Scholarlink

<p>In the self-distillation paradigm, collaborative learning strengthens self-teacher representations by leveraging the structural diversity of network branches. Unlike such mechanisms, our method augments the diversified expressiveness of self-teacher via collaborative learning of historical state information. Current history-based self-distillation approaches typically employ only a single historical state as the soft target to regularize student optimization, while neglecting the beneficial training effects stemming from discrepancies among multiple historical cues. To alleviate this limitation, we propose a novel Historical Collaborative Learning (HCL) framework empowered by adaptive dual-state historical supervision. Specifically, HCL jointly exploits long-term and short-term historical knowledge to produce complementary supervision signals. Considering the varying guidance of historical information across individual samples and training phases, an adaptive state selection mechanism is devised to dynamically identify optimal soft targets. The integration of heterogeneous historical constraints substantially enhances the network’s generalization and discriminative capabilities. Extensive evaluations on four representative image classification benchmarks, including CIFAR-100, TinyImageNet, ImageNet, and the fine-grained CUB-200-2011, validate that the proposed HCL achieves compelling performance in both effectiveness and efficiency against state-of-the-art methods. The source codes are available at <a href="https://github.com/zhang-xhxh/HCL.">https://github.com/zhang-xhxh/HCL.</a></p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Self-distillation framework via historical collaborative learning for image classification

  • 期刊论文

  • 发表日期：

    2026年7月15日

查看全文
  • Xiaohong Zhang,
  • Jianwen Xiang

摘要

In the self-distillation paradigm, collaborative learning strengthens self-teacher representations by leveraging the structural diversity of network branches. Unlike such mechanisms, our method augments the diversified expressiveness of self-teacher via collaborative learning of historical state information. Current history-based self-distillation approaches typically employ only a single historical state as the soft target to regularize student optimization, while neglecting the beneficial training effects stemming from discrepancies among multiple historical cues. To alleviate this limitation, we propose a novel Historical Collaborative Learning (HCL) framework empowered by adaptive dual-state historical supervision. Specifically, HCL jointly exploits long-term and short-term historical knowledge to produce complementary supervision signals. Considering the varying guidance of historical information across individual samples and training phases, an adaptive state selection mechanism is devised to dynamically identify optimal soft targets. The integration of heterogeneous historical constraints substantially enhances the network’s generalization and discriminative capabilities. Extensive evaluations on four representative image classification benchmarks, including CIFAR-100, TinyImageNet, ImageNet, and the fine-grained CUB-200-2011, validate that the proposed HCL achieves compelling performance in both effectiveness and efficiency against state-of-the-art methods. The source codes are available at https://github.com/zhang-xhxh/HCL.