Scholarlink

<p>Federated learning is a popular distributed learning paradigm in machine learning. Meanwhile, composition optimization is an effective hierarchical learning model, which appears in many machine learning applications such as meta learning and robust learning. More recently, although some federated composition optimization algorithms have been proposed, they still suffer from high sample and communication complexities. In the paper, thus, we propose a class of faster adaptive federated compositional optimization algorithms (i.e., MFCGD and AdaMFCGD) to solve the nonconvex distributed composition problems, which builds on the momentum-based variance reduced and local-SGD techniques. In particular, our adaptive algorithm (i.e., AdaMFCGD) uses a unified adaptive matrix to flexibly incorporate various adaptive learning rates. Moreover, we provide a solid theoretical analysis for our algorithms under non-i.i.d. setting, and prove our algorithms obtain a lower sample and communication complexities simultaneously than the existing federated composition optimization algorithms. Specifically, our algorithms obtain lower sample complexity of <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\({\tilde{O}}(\epsilon ^{-3})\)</EquationSource> </InlineEquation> with lower communication complexity of <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\({\tilde{O}}(\epsilon ^{-2})\)</EquationSource> </InlineEquation> in finding an <InlineEquation ID="IEq3"> <EquationSource Format="TEX">\(\epsilon\)</EquationSource> </InlineEquation>-stationary solution. We conduct numerical experiments on robust federated learning and distributed meta learning tasks to demonstrate the efficiency of our algorithms.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Faster Adaptive Momentum-Based Federated Methods for Distributed Composition Optimization

  • 期刊论文

  • 发表日期：

    2026年4月29日

查看全文
  • Feihu Huang,
  • Xinrui Wang,
  • Siqi Zhang,
  • Soncan Chen

摘要

Federated learning is a popular distributed learning paradigm in machine learning. Meanwhile, composition optimization is an effective hierarchical learning model, which appears in many machine learning applications such as meta learning and robust learning. More recently, although some federated composition optimization algorithms have been proposed, they still suffer from high sample and communication complexities. In the paper, thus, we propose a class of faster adaptive federated compositional optimization algorithms (i.e., MFCGD and AdaMFCGD) to solve the nonconvex distributed composition problems, which builds on the momentum-based variance reduced and local-SGD techniques. In particular, our adaptive algorithm (i.e., AdaMFCGD) uses a unified adaptive matrix to flexibly incorporate various adaptive learning rates. Moreover, we provide a solid theoretical analysis for our algorithms under non-i.i.d. setting, and prove our algorithms obtain a lower sample and communication complexities simultaneously than the existing federated composition optimization algorithms. Specifically, our algorithms obtain lower sample complexity of \({\tilde{O}}(\epsilon ^{-3})\) with lower communication complexity of \({\tilde{O}}(\epsilon ^{-2})\) in finding an \(\epsilon\) -stationary solution. We conduct numerical experiments on robust federated learning and distributed meta learning tasks to demonstrate the efficiency of our algorithms.