Scholarlink

<p>Catastrophic forgetting remains a critical challenge in lifelong learning where neural networks severely degrade on previous tasks when learning new ones. This paper proposes the first systematic integration of neural ordinary differential equations (Neural ODEs) with memory-augmented transformers to address this limitation. Neural ODEs enable smooth representation learning through continuous-time dynamics, while memory-augmented transformers provide explicit knowledge consolidation through attention-based retrieval. We derive rigorous PAC-learning theoretical bounds characterizing the relationships between model capacity, task sequence length, and forgetting severity, complemented by an adaptive memory management strategy. Comprehensive experiments on Split CIFAR-100, Permuted MNIST, and CORe50 demonstrate substantial improvements: 24% forgetting reduction and 10.3% accuracy gain over state-of-the-art methods. This work provides principled foundations for scalable lifelong learning systems capable of continuous knowledge accumulation without destructive interference.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Mitigating catastrophic forgetting in lifelong learning: a hybrid architecture integrating neural ordinary differential equations with memory-augmented transformers

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年12月15日

查看全文
  • Song Zhou,
  • Qiang Li

摘要

Catastrophic forgetting remains a critical challenge in lifelong learning where neural networks severely degrade on previous tasks when learning new ones. This paper proposes the first systematic integration of neural ordinary differential equations (Neural ODEs) with memory-augmented transformers to address this limitation. Neural ODEs enable smooth representation learning through continuous-time dynamics, while memory-augmented transformers provide explicit knowledge consolidation through attention-based retrieval. We derive rigorous PAC-learning theoretical bounds characterizing the relationships between model capacity, task sequence length, and forgetting severity, complemented by an adaptive memory management strategy. Comprehensive experiments on Split CIFAR-100, Permuted MNIST, and CORe50 demonstrate substantial improvements: 24% forgetting reduction and 10.3% accuracy gain over state-of-the-art methods. This work provides principled foundations for scalable lifelong learning systems capable of continuous knowledge accumulation without destructive interference.