<p>Transformers have revolutionized natural language processing and have recently demonstrated their potential in computer vision. They have shown promising results over convolutional neural networks in fundamental computer vision tasks. Despite their success, the inner working mechanism of vision transformers (ViTs) remain vague, highlighting the growing need for effective explainability techniques. This study addresses this gap by exploring different explainability methods tailored specifically to ViTs. We propose a structured taxonomy that categorizes these methods based on their underlying motivations, architectural designs, and application contexts. A comprehensive review of evaluation criteria, tools, and frameworks for assessing explainability in ViTs is also presented. Furthermore, we conducted experimental evaluations to analyze the performance and applicability of various explainability methods. The paper concludes by highlighting critical but unexplored aspects that could enhance the explainability of ViTs and suggests promising directions for future research.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Explainability of vision transformers: a comprehensive review and new perspectives

  • Rojina Kashefi,
  • Leili Barekatain,
  • Mohammad Sabokrou,
  • Fatemeh Aghaeipoor

摘要

Transformers have revolutionized natural language processing and have recently demonstrated their potential in computer vision. They have shown promising results over convolutional neural networks in fundamental computer vision tasks. Despite their success, the inner working mechanism of vision transformers (ViTs) remain vague, highlighting the growing need for effective explainability techniques. This study addresses this gap by exploring different explainability methods tailored specifically to ViTs. We propose a structured taxonomy that categorizes these methods based on their underlying motivations, architectural designs, and application contexts. A comprehensive review of evaluation criteria, tools, and frameworks for assessing explainability in ViTs is also presented. Furthermore, we conducted experimental evaluations to analyze the performance and applicability of various explainability methods. The paper concludes by highlighting critical but unexplored aspects that could enhance the explainability of ViTs and suggests promising directions for future research.