Scholarlink

In this paper, we propose a novel multimodal classification algorithm for pulmonary disease by combining chest X-ray images and clinical symptom texts. Our method leverages a Vision Transformer (ViT) to classify chest X-ray images, and a local eXtreme Gradient Boosting (XGB) model trained on the k nearest neighbors of clinical symptom data. The symptom texts are represented using a Bag-of-Words (BoW) model. Experiments on a real-world dataset collected from Chau Doc Hospital (An Giang, Vietnam) show that our multimodal model achieves a classification accuracy of 85.29% on the testset. Furthermore, we introduce the LoXIM (Local eXplanation with Instance-based Modeling), explains the prediction for each test case by extracting important features (words) from the local XGB model. For each test symptom text, words that match the important features are highlighted according to their importance weights. This enhances transparency and enables clinicians to better understand and trust the model’s decisions.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Explainable AI for Lung Disease Classification: Looking Inside the Black Box

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Thi-Diem Truong,
  • Thanh-Nghi Do

摘要

In this paper, we propose a novel multimodal classification algorithm for pulmonary disease by combining chest X-ray images and clinical symptom texts. Our method leverages a Vision Transformer (ViT) to classify chest X-ray images, and a local eXtreme Gradient Boosting (XGB) model trained on the k nearest neighbors of clinical symptom data. The symptom texts are represented using a Bag-of-Words (BoW) model. Experiments on a real-world dataset collected from Chau Doc Hospital (An Giang, Vietnam) show that our multimodal model achieves a classification accuracy of 85.29% on the testset. Furthermore, we introduce the LoXIM (Local eXplanation with Instance-based Modeling), explains the prediction for each test case by extracting important features (words) from the local XGB model. For each test symptom text, words that match the important features are highlighted according to their importance weights. This enhances transparency and enables clinicians to better understand and trust the model’s decisions.