Federated learning (FL), as an emerging distributed machine learning framework, aims to improve model training efficiency while preserving data privacy. However, its effectiveness is hindered by challenges such as system heterogeneity, statistical heterogeneity, and constrained communication resources. To address these issues, we propose a heterogeneity-aware semi-asynchronous federated learning framework, named HASA-Fed, which incorporates an adjustable time threshold-based aggregation mechanism and a heterogeneity-aware scheduling strategy. Specifically, HASA-Fed synergizes synchronous and asynchronous FL paradigms by employing a dynamic time threshold as the criterion for server aggregation, which adaptively adjusts according to the training progress of devices. Simultaneously, the framework integrates heterogeneity-aware scheduling, where the system selects a subset of devices to upload local updates prior to each aggregation. This selection is guided by both system heterogeneity (e.g., computational capabilities) and statistical heterogeneity (e.g., data distribution), under a predefined scheduling rate, thereby alleviating uplink communication overhead. These two components collectively enhance the training efficiency of federated learning. The efficacy of HASA-Fed is validated through extensive experiments on various benchmark models and datasets, demonstrating superior performance compared to state-of-the-art FL aggregation mechanisms and scheduling algorithms.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Heterogeneity-Aware Semi-asynchronous Federated Learning

  • Junying He,
  • Yuxiang Chen,
  • Jigang Wen,
  • Kun Xie,
  • Kan Yang,
  • Xiaoyan Chen,
  • Tianxiong Liu

摘要

Federated learning (FL), as an emerging distributed machine learning framework, aims to improve model training efficiency while preserving data privacy. However, its effectiveness is hindered by challenges such as system heterogeneity, statistical heterogeneity, and constrained communication resources. To address these issues, we propose a heterogeneity-aware semi-asynchronous federated learning framework, named HASA-Fed, which incorporates an adjustable time threshold-based aggregation mechanism and a heterogeneity-aware scheduling strategy. Specifically, HASA-Fed synergizes synchronous and asynchronous FL paradigms by employing a dynamic time threshold as the criterion for server aggregation, which adaptively adjusts according to the training progress of devices. Simultaneously, the framework integrates heterogeneity-aware scheduling, where the system selects a subset of devices to upload local updates prior to each aggregation. This selection is guided by both system heterogeneity (e.g., computational capabilities) and statistical heterogeneity (e.g., data distribution), under a predefined scheduling rate, thereby alleviating uplink communication overhead. These two components collectively enhance the training efficiency of federated learning. The efficacy of HASA-Fed is validated through extensive experiments on various benchmark models and datasets, demonstrating superior performance compared to state-of-the-art FL aggregation mechanisms and scheduling algorithms.