Scholarlink

<p>In the realm of federated learning, a collaborative yet decentralized approach to machine learning, communication efficiency is a critical concern, particularly under constraints of limited bandwidth and resources. This paper evaluates FedSNIP, a novel method that leverages the SNIP (Single-shot Network Pruning based on Connection Sensitivity) technique within this context. By utilizing SNIP, FedSNIP effectively prunes neural networks, converting numerous weights to zero and resulting in sparser weight representations. This substantial reduction in weight density significantly decreases the volume of parameters that need to be communicated to the server, thereby reducing the communication overhead. Our experiments on the CIFAR-10 and UCI-HAR dataset demonstrate that FedSNIP not only lowers the data transmission between clients and the server but also maintains competitive model accuracy, comparable to conventional federated learning models. Additionally, we analyze various compression algorithms applied after pruning, specifically evaluating the compressed sparse row, coordinate list, and compressed sparse column formats to identify the most efficient approach. Our results show that compressed sparse row not only compresses the data more effectively and quickly but also achieves the highest reduction in data size, making it the most suitable format for enhancing the efficiency of federated learning, particularly in scenarios with restricted communication capabilities.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Reducing communication overhead through one-shot model pruning in federated learning

  • 期刊论文

  • 发表日期：

    2025年5月5日

查看全文
  • Rómulo Bustincio,
  • Allan M. de Souza,
  • Joahannes B. D. da Costa,
  • Luis F. G. Gonzalez,
  • Luiz F. Bittencourt

摘要

In the realm of federated learning, a collaborative yet decentralized approach to machine learning, communication efficiency is a critical concern, particularly under constraints of limited bandwidth and resources. This paper evaluates FedSNIP, a novel method that leverages the SNIP (Single-shot Network Pruning based on Connection Sensitivity) technique within this context. By utilizing SNIP, FedSNIP effectively prunes neural networks, converting numerous weights to zero and resulting in sparser weight representations. This substantial reduction in weight density significantly decreases the volume of parameters that need to be communicated to the server, thereby reducing the communication overhead. Our experiments on the CIFAR-10 and UCI-HAR dataset demonstrate that FedSNIP not only lowers the data transmission between clients and the server but also maintains competitive model accuracy, comparable to conventional federated learning models. Additionally, we analyze various compression algorithms applied after pruning, specifically evaluating the compressed sparse row, coordinate list, and compressed sparse column formats to identify the most efficient approach. Our results show that compressed sparse row not only compresses the data more effectively and quickly but also achieves the highest reduction in data size, making it the most suitable format for enhancing the efficiency of federated learning, particularly in scenarios with restricted communication capabilities.