Scholarlink

Federated Learning (FL) is a distributed learning paradigm which entails the training of Machine Learning (ML) models across multiple computing devices, while keeping the training data local to the devices. One of the key challenges of FL is heterogeneity of both the computing devices and data. This challenge might ultimately lead to the FL model instability, slow convergence, and performance degradation. This work introduces Adaptive FedProx, a new FedProx algorithm extension that dynamically modifies its proximal regularisation term in response to real-time heterogeneity detection. In order to direct adaptive regularisation, we present the Heterogeneity-Aware Performance Index (HAPI), a metric that measures the difference between local and global models. We uncover an important trade-off through extensive experiments on CIFAR-10 across Independent and Identically Distributed (IID), mild non-IID, and strong non-IID scenarios: Adaptive FedProx exhibits superior robustness to data heterogeneity, despite a 1.6% performance decrease in homogeneous (IID) settings when compared to FedAvg (87.27% vs. 88.87%, \(p<0.001\) ). When moving from IID to strong non-IID data, Adaptive FedProx shows 3.5% better robustness with a performance drop of only 22.8% versus FedAvg’s 26.3%, and it achieves 67.36% accuracy in strong non-IID scenarios compared to FedAvg’s 65.35%. These results imply that, at the expense of a minor drop in performance in homogeneous environments, adaptive regularisation techniques can improve federated learning’s resistance to heterogeneous data distributions.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

HAPI—FedProx: Heterogeneity—Aware Adaptive Proximal Optimization for Federated Learning

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Kudzaishe Mhou,
  • Makhamisa Senekane

摘要

Federated Learning (FL) is a distributed learning paradigm which entails the training of Machine Learning (ML) models across multiple computing devices, while keeping the training data local to the devices. One of the key challenges of FL is heterogeneity of both the computing devices and data. This challenge might ultimately lead to the FL model instability, slow convergence, and performance degradation. This work introduces Adaptive FedProx, a new FedProx algorithm extension that dynamically modifies its proximal regularisation term in response to real-time heterogeneity detection. In order to direct adaptive regularisation, we present the Heterogeneity-Aware Performance Index (HAPI), a metric that measures the difference between local and global models. We uncover an important trade-off through extensive experiments on CIFAR-10 across Independent and Identically Distributed (IID), mild non-IID, and strong non-IID scenarios: Adaptive FedProx exhibits superior robustness to data heterogeneity, despite a 1.6% performance decrease in homogeneous (IID) settings when compared to FedAvg (87.27% vs. 88.87%, \(p<0.001\) ). When moving from IID to strong non-IID data, Adaptive FedProx shows 3.5% better robustness with a performance drop of only 22.8% versus FedAvg’s 26.3%, and it achieves 67.36% accuracy in strong non-IID scenarios compared to FedAvg’s 65.35%. These results imply that, at the expense of a minor drop in performance in homogeneous environments, adaptive regularisation techniques can improve federated learning’s resistance to heterogeneous data distributions.