MiniMedMind: Small-Scale LLM for High-Performance Medical VQA on Chest X-ray
摘要
Visual Question Answering (VQA) in medical imaging has the potential to transform clinical practice by enabling automated, image-based responses to medical queries, particularly in radiology. However, most Medical VQA models are large-scale and computationally demanding, making them impractical for widespread use in resource-limited healthcare settings. In this work, we introduce MiniMedMind, a lightweight and efficient VQA model designed to perform medical question-answering tasks on chest X-Ray (CXR) images. MiniMedMind combines a vision encoder, a projector layer, and a fine-tuned small-scale language model to achieve robust performance with minimal computational requirements. The vision encoder leverages pretrained weights from CheXAgent for effective feature extraction, while the projector aligns these features with a fine-tuned Llama 3.2 model (3B parameters) enhanced with Low-Rank Adaptation (LoRA) to generate clinically accurate responses. The training dataset combines patient-doctor conversations, MIMIC-CXR data, and synthetic conversations generated by GPT-3.5, enabling MiniMedMind to effectively interpret and respond to medical queries. Evaluated on report generation and VQA tasks, MiniMedMind performs near top models like XrayGPT and Med-MoE, achieving competitive accuracy with a significantly lighter architecture. These results position MiniMedMind as an efficient, effective solution for resource-constrained medical AI applications. MiniMedMind offers a computationally efficient solution for Medical VQA, enabling AI-driven diagnostic support that can be applied in diverse clinical and educational settings.