Hybrid federated learning with prototype alignment and adaptive distillation
摘要
To address the performance degradation and increased communication overhead that often arise in federated learning under dual heterogeneity of data distributions and model architectures, we propose FedPAAD, a hybrid federated learning framework based on prototype alignment and adaptive distillation. Methodologically: (1) we introduce a reversible dynamic projection for heterogeneous backbones, coupled with Hessian spectrum–guided dimensional adaptation, to map local features into a shared space for robust class-prototype computation and cross-client alignment; (2) on the head network, we employ bidirectional knowledge distillation–using global knowledge to mitigate distribution shift while preserving personalization via local “dark knowledge”; and (3) we design a two-stage weight scheduling mechanism further modulated by distribution divergence, emphasizing prototype learning in early rounds and distillation in later rounds.Theoretically, under a non-convex setting with bounded variance and explicitly accounting for errors from projection and low-rank compression, we derive an