Scholarlink

As a distributed learning paradigm, federated learning (FL) can protect data privacy and achieve distributed collaborative training, but the noise labels in the client data will seriously impair the model performance. Existing methods rely on a large number of clean clients for pre-training to correct noise samples, but in actual scenarios, the number of clean clients may not be enough to meet the requirements. To this end, we propose the FedSkd framework: in the first stage, the local eigendimension (LID) features of the client are extracted, and the noise/clean client classification is realized by combining with the Gaussian mixture model (GMM); In the second stage, stable pre-training was performed on the screened clean client in combination with knowledge distillation (KD), and the noise samples were iteratively corrected by a progressive label correction mechanism. Experiments show that the model trained by FedSkd in the scenario of low proportion of clean clients ( \(<\) 30%) still maintains good robustness.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Knowledge Distillation for Federated Learning with Many Noisy Clients

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Liang Peng,
  • Zoufeng Jiang,
  • Huilin Li,
  • Yubo Yang,
  • Tianxiong Liu

摘要

As a distributed learning paradigm, federated learning (FL) can protect data privacy and achieve distributed collaborative training, but the noise labels in the client data will seriously impair the model performance. Existing methods rely on a large number of clean clients for pre-training to correct noise samples, but in actual scenarios, the number of clean clients may not be enough to meet the requirements. To this end, we propose the FedSkd framework: in the first stage, the local eigendimension (LID) features of the client are extracted, and the noise/clean client classification is realized by combining with the Gaussian mixture model (GMM); In the second stage, stable pre-training was performed on the screened clean client in combination with knowledge distillation (KD), and the noise samples were iteratively corrected by a progressive label correction mechanism. Experiments show that the model trained by FedSkd in the scenario of low proportion of clean clients ( \(<\) 30%) still maintains good robustness.