Federated learning is a decentralized machine learning scheme, which allows cooperative training of models on a multi-device scale without the exchange of personal information, thus privacy is not violated, and it adheres to data protection laws. Non-IID data distributions, curtailed communication bandwidth, and heterogeneous device abilities however are some of the major challenges that federated learning encounters during hyperparameter tuning. The factors prevent convergence, performance, and generalization of the model. This paper aims to solve these problems by presenting a meta-learning-based hyperparameter tuning framework that uses the existing knowledge about clients to better adapt to novel data distributions. The method reduces both overhead and diversity in communication as well as improves personalization and convergence. Our meta-learning strategies decrease the number of convergence rounds by a factor of 27, final accuracy by up to 6.4 percent and decreases client update variance, all showing considerable efficiency and resilience benefits in federated learning, as compared to FedAvg and Bayesian optimization.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Efficient Hyperparameter Tuning in Federated Learning Systems Using Meta-learning

  • Eshaan Saha,
  • Vidit Singh,
  • Malvinder Singh Bali

摘要

Federated learning is a decentralized machine learning scheme, which allows cooperative training of models on a multi-device scale without the exchange of personal information, thus privacy is not violated, and it adheres to data protection laws. Non-IID data distributions, curtailed communication bandwidth, and heterogeneous device abilities however are some of the major challenges that federated learning encounters during hyperparameter tuning. The factors prevent convergence, performance, and generalization of the model. This paper aims to solve these problems by presenting a meta-learning-based hyperparameter tuning framework that uses the existing knowledge about clients to better adapt to novel data distributions. The method reduces both overhead and diversity in communication as well as improves personalization and convergence. Our meta-learning strategies decrease the number of convergence rounds by a factor of 27, final accuracy by up to 6.4 percent and decreases client update variance, all showing considerable efficiency and resilience benefits in federated learning, as compared to FedAvg and Bayesian optimization.