Visual Question Answering to Generate Case-Based Explanations for Image Classification
摘要
To clarify how artificial intelligence models for image classification obtain their predictions, instance-based explanations give examples of other predictions obtained by those models for similar images. XAI (eXplainable Artificial Intelligence) techniques find those examples by getting images similar to the query using similarity metrics that consider image features. In this work, we introduce a Case-Based Reasoning approach that obtains these types of explanations (explanations by examples and counterfactuals) using Visual Question Answering techniques that take advantage of the high performance achieved by Large Language Models. We have evaluated our approach through an offline evaluation, in a context where images to classify and explain may be low quality. We obtained promising results and confirmed the benefits of using multimodal models in XAI, combining knowledge focused on images and natural language.