Scholarlink

<p>Federated learning (FL) has emerged as a promising framework for distributed machine learning, allowing edge devices to collaboratively train models without sharing local data. However, communication efficiency remains a critical bottleneck. This paper proposes a novel framework that integrates regularized sparse randomized networks to address this challenge. The core idea is to introduce a global probability mask, updated iteratively, that guides the formation of sparse sub-networks on edge devices. Each device optimizes local sub-networks by sampling binary masks based on the global probability mask, effectively reducing model complexity while maintaining predictive performance. The proposed method incorporates a regularization term into the loss function to enhance sparsity and mitigate redundancy in the transmitted binary masks. This approach balances the trade-off between communication overhead on the natural language query (NLQ) based tasks and model learning performance, significantly reducing bitrate requirements in uplink communication. Experimental results on the MNIST dataset, under both IID and non-IID settings, demonstrate the effectiveness of the proposed framework. Notably, the proposed method achieves comparable test accuracy to the existing approaches, such as FedPM, while reducing communication bitrate by up to 80% in certain configurations. Moreover, the analysis of different regularization parameters reveals a clear trade-off between the sparsity and accuracy, where a lower regularization can enhance convergence and generalization, achieving higher accuracy in fewer communication rounds, while higher values prioritize communication efficiency.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Task-oriented efficient communication in federated learning via regularized sparse randomized networks and NLQ

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年11月26日

查看全文
  • Liyuan Zhang,
  • Zhaoli Chen,
  • Zhipeng Meng,
  • Xuhua Ai,
  • Qi Meng

摘要

Federated learning (FL) has emerged as a promising framework for distributed machine learning, allowing edge devices to collaboratively train models without sharing local data. However, communication efficiency remains a critical bottleneck. This paper proposes a novel framework that integrates regularized sparse randomized networks to address this challenge. The core idea is to introduce a global probability mask, updated iteratively, that guides the formation of sparse sub-networks on edge devices. Each device optimizes local sub-networks by sampling binary masks based on the global probability mask, effectively reducing model complexity while maintaining predictive performance. The proposed method incorporates a regularization term into the loss function to enhance sparsity and mitigate redundancy in the transmitted binary masks. This approach balances the trade-off between communication overhead on the natural language query (NLQ) based tasks and model learning performance, significantly reducing bitrate requirements in uplink communication. Experimental results on the MNIST dataset, under both IID and non-IID settings, demonstrate the effectiveness of the proposed framework. Notably, the proposed method achieves comparable test accuracy to the existing approaches, such as FedPM, while reducing communication bitrate by up to 80% in certain configurations. Moreover, the analysis of different regularization parameters reveals a clear trade-off between the sparsity and accuracy, where a lower regularization can enhance convergence and generalization, achieving higher accuracy in fewer communication rounds, while higher values prioritize communication efficiency.