Explainable Vision Transformers for Real-Time Micro-expression Detection
摘要
Micro-expressions, transitory expressions revealing feelings, have posed a challenge for computer vision. Although convolutional neural network (CNN)-based models were popular for facial emotion classification, motor-based models have shown better performance in computer vision tasks. The recent emergence of Vision Transformers, with their exceptional feature learning capabilities, presents a promising opportunity. In this paper, we also compare the performance of Vision Transformers with other models, such as traditional convolutional neural networks, YoloV8, MobileNet, and AlexNet. This work leverages this potential by proposing an Explainable Vision Transformer for micro-expression detection. The Explainable Vision Transformer leverages the power of Vision Transformers, which have been employed since their introduction to extract informative features from facial images. To gain insight into the model’s decision-making process, we incorporated explainability techniques, including LIME, SHAP, Grad-CAM, saliency maps, and integrated gradients. The Explainable Vision Transformer architecture, along with the intricate feature extraction within the network, is presented in detail. We demonstrate the effectiveness of our approach through experiments and analysis of XAI results, revealing the model’s focus and explanations for each micro-expression class. Finally, we examine its results when deployed in a live system that predicts real-time micro-expressions.