Scholarlink

<p>Federated and Continual Learning have emerged as promising paradigms for the privacy-aware use of Deep Learning in dynamic environments by addressing spatial and temporal constraints on data availability. However, Client Drift and Catastrophic Forgetting are fundamental obstacles to ensuring robust performance. Existing work only addresses these problems separately, neglecting the fact that the root cause behind them, namely an unexpected shift in the data distribution, is connected. We propose a unified analysis framework for building a controlled test environment where we can jointly model spatial and temporal shifts, more closely emulating real dynamic settings. By generating a 3D landscape of the combined performance impact, we show that a moderate combination of both shifts can even improve the performance of the resulting model (“Generalization Bump”). We apply a simple and commonly used method from continual learning in the federated setting and observe this reoccurring phenomenon.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Jointly exploring client drift and catastrophic forgetting in dynamic learning

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年2月18日

查看全文
  • Niklas Babendererde,
  • Moritz Fuchs,
  • Camila Gonzalez,
  • Yuri Tolkach,
  • Anirban Mukhopadhyay

摘要

Federated and Continual Learning have emerged as promising paradigms for the privacy-aware use of Deep Learning in dynamic environments by addressing spatial and temporal constraints on data availability. However, Client Drift and Catastrophic Forgetting are fundamental obstacles to ensuring robust performance. Existing work only addresses these problems separately, neglecting the fact that the root cause behind them, namely an unexpected shift in the data distribution, is connected. We propose a unified analysis framework for building a controlled test environment where we can jointly model spatial and temporal shifts, more closely emulating real dynamic settings. By generating a 3D landscape of the combined performance impact, we show that a moderate combination of both shifts can even improve the performance of the resulting model (“Generalization Bump”). We apply a simple and commonly used method from continual learning in the federated setting and observe this reoccurring phenomenon.