Scholarlink

<p>Nowadays deep neural networks have surpassed human recognition abilities on significant image datasets. However, due to the limitations and biases in training samples, post hoc explanation methods reveal that models rely on background information for decision-making, regardless of the correctness of the decisions. Models can maintain stable performance under test distribution shifts only if they learn causally relevant features of the target from the training data. To address distributional shift problems, out-of-distribution generalization approaches improve model generalization on unseen domains based on domain label information. Recently, the emerging paradigm of explanation-guided learning, which integrates explanations into model training, has significantly reduced the learning of spurious correlations and effectively improved model generalization on unseen distributions, performing comparably to related out-of-distribution generalization methods. Explanation-guided learning primarily incorporates prior knowledge or bounding boxes. However, explanation-guided learning with prior knowledge can still focus on spurious features, and using bounding boxes as ground truth incurs high manual annotation costs. To address these issues, we divide the image dataset into foreground-dominant and background-dominant subsets. For the foreground-dominant dataset, we propose an <i>Annotation-based Explanation Consistency (AEC)</i> method to constrain the model to focus on the most discriminative foreground regions. For the background-dominant dataset, we propose a <i>Class-wise Feature Reweighting Explanation Consistency (CFWEC)</i> method to guide the model in learning causally relevant features. We validate the effectiveness of these methods on both foreground-dominant and background-dominant datasets, demonstrating that they effectively reduce the learning of spurious correlations. We also analyze the current limitations of the method in terms of manual annotation and hyperparameter tuning, while discussing prospective solutions involving high-performance computing or distributed platforms for implementing automated model annotation and distributed training in future work.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Directing model attention to the discriminative foreground features

  • 期刊论文

  • 发表日期：

    2025年8月12日

查看全文
  • Jian-Xun Mi,
  • Lu Pan,
  • Qiang Fu,
  • Shenglei Shi,
  • Tao Chen,
  • Xiao Cheng

摘要

Nowadays deep neural networks have surpassed human recognition abilities on significant image datasets. However, due to the limitations and biases in training samples, post hoc explanation methods reveal that models rely on background information for decision-making, regardless of the correctness of the decisions. Models can maintain stable performance under test distribution shifts only if they learn causally relevant features of the target from the training data. To address distributional shift problems, out-of-distribution generalization approaches improve model generalization on unseen domains based on domain label information. Recently, the emerging paradigm of explanation-guided learning, which integrates explanations into model training, has significantly reduced the learning of spurious correlations and effectively improved model generalization on unseen distributions, performing comparably to related out-of-distribution generalization methods. Explanation-guided learning primarily incorporates prior knowledge or bounding boxes. However, explanation-guided learning with prior knowledge can still focus on spurious features, and using bounding boxes as ground truth incurs high manual annotation costs. To address these issues, we divide the image dataset into foreground-dominant and background-dominant subsets. For the foreground-dominant dataset, we propose an Annotation-based Explanation Consistency (AEC) method to constrain the model to focus on the most discriminative foreground regions. For the background-dominant dataset, we propose a Class-wise Feature Reweighting Explanation Consistency (CFWEC) method to guide the model in learning causally relevant features. We validate the effectiveness of these methods on both foreground-dominant and background-dominant datasets, demonstrating that they effectively reduce the learning of spurious correlations. We also analyze the current limitations of the method in terms of manual annotation and hyperparameter tuning, while discussing prospective solutions involving high-performance computing or distributed platforms for implementing automated model annotation and distributed training in future work.