To efficiently and securely train large models on resource-constrained devices, a Split Federated Learning (SFL) framework has been proposed. However, existing execution strategies of SFL have inherent drawbacks. One approach enables one-to-one parallel communication between clients and the server but requires multiple model replicas on the server, leading to excessive memory consumption. The other approach eliminates the need for multiple model copies but processes client tasks sequentially, causing task queuing and delays. Additionally, due to dependency constraints within model computations, server-side idle waiting becomes a bottleneck, a challenge also present in SFL. To enhance the parallel efficiency of SFL, we propose the PSFL framework. The framework partitions the server cluster into groups and allocates server models accordingly by group. Furthermore, PSFL enables parallel training on the server by further splitting the server model and scheduling different client data for concurrent computation. We compare the PSFL framework with other split federated learning algorithms on four datasets. Experimental results demonstrate that PSFL achieves faster convergence without sacrificing accuracy, improving average per-round training time by 17%–32%.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Group-Based Parallel Split Federated Learning

  • Xiaofan Zhou,
  • Xiangqi Xiao,
  • Yuxiang Chen,
  • Jigang Wen,
  • Kun Xie,
  • Kan Yang,
  • Tianxiong Liu

摘要

To efficiently and securely train large models on resource-constrained devices, a Split Federated Learning (SFL) framework has been proposed. However, existing execution strategies of SFL have inherent drawbacks. One approach enables one-to-one parallel communication between clients and the server but requires multiple model replicas on the server, leading to excessive memory consumption. The other approach eliminates the need for multiple model copies but processes client tasks sequentially, causing task queuing and delays. Additionally, due to dependency constraints within model computations, server-side idle waiting becomes a bottleneck, a challenge also present in SFL. To enhance the parallel efficiency of SFL, we propose the PSFL framework. The framework partitions the server cluster into groups and allocates server models accordingly by group. Furthermore, PSFL enables parallel training on the server by further splitting the server model and scheduling different client data for concurrent computation. We compare the PSFL framework with other split federated learning algorithms on four datasets. Experimental results demonstrate that PSFL achieves faster convergence without sacrificing accuracy, improving average per-round training time by 17%–32%.