Explainability of vision transformers: a comprehensive review and new perspectives
摘要
Transformers have revolutionized natural language processing and have recently demonstrated their potential in computer vision. They have shown promising results over convolutional neural networks in fundamental computer vision tasks. Despite their success, the inner working mechanism of vision transformers (ViTs) remain vague, highlighting the growing need for effective explainability techniques. This study addresses this gap by exploring different explainability methods tailored specifically to ViTs. We propose a structured taxonomy that categorizes these methods based on their underlying motivations, architectural designs, and application contexts. A comprehensive review of evaluation criteria, tools, and frameworks for assessing explainability in ViTs is also presented. Furthermore, we conducted experimental evaluations to analyze the performance and applicability of various explainability methods. The paper concludes by highlighting critical but unexplored aspects that could enhance the explainability of ViTs and suggests promising directions for future research.