Scholarlink

<p>Empirical Risk Minimization (ERM) models often rely on spurious correlations between features and labels during the learning process, leading to shortcut learning behavior that undermines robustness generalization performance. Current research mainly targets identifying or mitigating a single shortcut; however, in real-world scenarios, cues within the data are diverse and unknown. In empirical studies, we reveal that models rely more on strong shortcuts than weak ones, with their performance under multiple shortcuts typically falling between that of an individual shortcut. To address these challenges, we propose <b>MiMu</b>, a novel method integrated with Transformer-based ERMs designed to <b>Mi</b>tigate <b>Mu</b>ltiple shortcut learning behavior, which incorporates self-calibration strategy and self-improvement strategy. In the source model, we first propose the self-calibration strategy to prevent the model from relying on shortcuts and make overconfident predictions. Then, we design self-improvement strategy in target model to further reduce the reliance on multiple shortcuts. The random mask strategy involves randomly masking partial attention positions to diversify the focus of target model avoiding fixation on a fixed region. Meanwhile, the adaptive attention alignment module facilitates the alignment of attention weights to the calibrated source model, without the need for post-hoc attention maps or supervision. Finally, extensive experiments conducted on Natural Language Processing (NLP) and Computer Vision (CV) demonstrate the effectiveness of MiMu in improving the robustness generalization abilities.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

MiMu: mitigating multiple shortcut learning behavior of transformers

  • 期刊论文

  • 发表日期：

    2025年10月28日

查看全文
  • Lili Zhao,
  • Qi Liu,
  • Wei Chen,
  • Liyi Chen,
  • Ruijun Sun,
  • Min Hou,
  • Yang Wang,
  • Shijin Wang,
  • Pingping Ren,
  • Jiafeng Zhou

摘要

Empirical Risk Minimization (ERM) models often rely on spurious correlations between features and labels during the learning process, leading to shortcut learning behavior that undermines robustness generalization performance. Current research mainly targets identifying or mitigating a single shortcut; however, in real-world scenarios, cues within the data are diverse and unknown. In empirical studies, we reveal that models rely more on strong shortcuts than weak ones, with their performance under multiple shortcuts typically falling between that of an individual shortcut. To address these challenges, we propose MiMu, a novel method integrated with Transformer-based ERMs designed to Mitigate Multiple shortcut learning behavior, which incorporates self-calibration strategy and self-improvement strategy. In the source model, we first propose the self-calibration strategy to prevent the model from relying on shortcuts and make overconfident predictions. Then, we design self-improvement strategy in target model to further reduce the reliance on multiple shortcuts. The random mask strategy involves randomly masking partial attention positions to diversify the focus of target model avoiding fixation on a fixed region. Meanwhile, the adaptive attention alignment module facilitates the alignment of attention weights to the calibrated source model, without the need for post-hoc attention maps or supervision. Finally, extensive experiments conducted on Natural Language Processing (NLP) and Computer Vision (CV) demonstrate the effectiveness of MiMu in improving the robustness generalization abilities.