Scholarlink

<p>Label skew is a widely discussed issue in federated learning. Common solutions often involve weighted parameter aggregation based on the scale of private data and the introduction of global signals to regularize the local model. However, under label skew conditions, local data exhibits diverse and imbalanced properties. Allocating high weight to clients with more samples can distract the global aggregation process. To make matters worse, global signals introduce an unavoidable computational burden. In this paper, we approach these issues from a novel over-confidence perspective. First, we prove that under severe label skew, local models exhibit greater over-confidence in private data, which appears as a smaller empirical loss owing to increased logits magnitudes. We leverage the local logits magnitude as a metric for private data balance to reweight parameter aggregation. Second, we point out that over-confidence in local data results in divergence in the aggregated classifier as different local classifiers bias distinct instance-rich classes. We address this by uniformly rescaling the normalized logits to detach the classifier weight norm from the optimization equation. This successfully calibrates the local model without relying on global signals. Empirical results on four datasets manifest the superiority of our proposed method. Our code is available at <a href="https://github.com/WenkeHuang/RevisitFL">https://github.com/WenkeHuang/RevisitFL</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Revisiting federated learning with label skew: an over-confidence perspective

  • 期刊论文

  • 发表日期：

    2025年7月4日

查看全文
  • Mang Ye,
  • Wenke Huang,
  • Zekun Shi,
  • He Li,
  • Bo Du

摘要

Label skew is a widely discussed issue in federated learning. Common solutions often involve weighted parameter aggregation based on the scale of private data and the introduction of global signals to regularize the local model. However, under label skew conditions, local data exhibits diverse and imbalanced properties. Allocating high weight to clients with more samples can distract the global aggregation process. To make matters worse, global signals introduce an unavoidable computational burden. In this paper, we approach these issues from a novel over-confidence perspective. First, we prove that under severe label skew, local models exhibit greater over-confidence in private data, which appears as a smaller empirical loss owing to increased logits magnitudes. We leverage the local logits magnitude as a metric for private data balance to reweight parameter aggregation. Second, we point out that over-confidence in local data results in divergence in the aggregated classifier as different local classifiers bias distinct instance-rich classes. We address this by uniformly rescaling the normalized logits to detach the classifier weight norm from the optimization equation. This successfully calibrates the local model without relying on global signals. Empirical results on four datasets manifest the superiority of our proposed method. Our code is available at https://github.com/WenkeHuang/RevisitFL.