As the aging population grows, physical hazards such as falls and slips in the home environment pose significant threats to elderly individuals’ safety. Traditional hazard detection methods often rely on centralized training with large labeled datasets, which raises concerns over privacy and data storage. Federated learning (FL), such as FedAvg and FedAdam, addresses these issues by enabling model training while keeping data decentralized. However, the traditional FedAdam algorithm aggregates model updates using a simple averaging method, which can lead to reduced accuracy when there is uneven data distribution or significant local model variation. This paper proposes an improved FedAdam algorithm based on Quadratic Mean (QM) for physical hazard detection in elderly home environments. The QM aggregation method enhances the stability and accuracy of model updates, improving overall performance. Additionally, Faster R-CNN, a state-of-the-art object detection model, is employed for hazard detection, enabling precise identification of risks like falls and slips. Experimental results show that the proposed method outperforms traditional approaches in terms of accuracy, efficiency, and robustness, providing a privacy-preserving, intelligent solution for real-time hazard detection in elderly home environments.