cFedLAD: A Clustered Additive LoRA Framework for Robust and Personalized Federated Learning
摘要
While parameter-efficient adaptation techniques help clients deploy and adapt large pre-trained models in Federated Learning (FL), they fall short in addressing cross-client collaboration under statistical heterogeneity. These methods often struggle to balance generalization with local personalization, resulting in unstable training and suboptimal performance. We propose Clustered Federated LoRA with Additive Decomposition (cFedLAD), a novel framework that integrates global and cluster-specific learning for adapting foundation models in heterogeneous FL environments. cFedLAD combines a shared global LoRA adapter, which captures common features across clients, with cluster-specific adapters that model residual variations within each group of similar clients. This additive design enables effective knowledge sharing while preserving meaningful local adaptation. Experiments on diverse federated benchmarks show that cFedLAD outperforms standard global and clustered LoRA baselines, achieving higher accuracy and faster convergence under non-IID conditions. These results highlight cFedLAD’s practical value for scalable, resource-efficient FL deployments.