A Variational Approach to Personalized Federated Learning and Its Improvement
摘要
Federated learning (FL) is a new approach that allows clients from different locations to work together on a global model without sharing raw data. However, training one standard model is not optimal for local optimization due to heterogeneity. Thus, personalized federated learning (PFL) paradigms aim to combine local and global information while preserving diversity among local clients. However, these methods require more theoretical motivation. This paper formalizes the unfired analysis form by defining a linear mapping on the server’s aggregation step for a deeper understanding of the implementation design and to improve existing methods further. Moreover, we show that the existing FL and PFL methods are specific cases in our framework with selected mapping matrices, including FedAvg, FedMTL, FedProx, MAML, and pure local training. Based on that, we propose a novel data-driven graph Laplacian-induced PFL scheme named GLIP-FL to address statistical heterogeneity issues and improve computation efficiency. We implement evaluation experiments of the proposed method on four popular data sets with five FL baselines in three data partitions, both homogeneous and heterogeneous, and the comprehensive experiments show the great potential of our method.