Scholarlink

Vision Transformer (ViT) becomes a very well-known model in computer vision due to its unique architecture, which uses self-attention mechanisms to effectively extract features. As a result, most of predictions from this model are correct. Even the performance is very high, it is crucial to understand the features the model relies on for making decision. Our work proposes a method for visualizing the features extracted by ViT by identifying Class Activation Maps (CAMs) at each head in the self-attention module. Then we evaluate the importance of each CAM and aggregate them to get the final CAM representation. In addition, we used evaluation metrics for XAI such as Localization metrics and Faithfulness metrics to assess whether our explanation method provides meaningful insights into the model’s decision-making process.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Vision Transformer Explainability: Enhancing Class Activation Map of Every Heads Using Confidence Scores with Perturbation

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Dai-Nien Tran,
  • Cao-Khoi Nguyen,
  • Trong-Nghia Pham

摘要

Vision Transformer (ViT) becomes a very well-known model in computer vision due to its unique architecture, which uses self-attention mechanisms to effectively extract features. As a result, most of predictions from this model are correct. Even the performance is very high, it is crucial to understand the features the model relies on for making decision. Our work proposes a method for visualizing the features extracted by ViT by identifying Class Activation Maps (CAMs) at each head in the self-attention module. Then we evaluate the importance of each CAM and aggregate them to get the final CAM representation. In addition, we used evaluation metrics for XAI such as Localization metrics and Faithfulness metrics to assess whether our explanation method provides meaningful insights into the model’s decision-making process.