This paper presents a novel multimodal approach to medical image analysis that combines state-of-the-art computer vision models with Large Language Models (LLMs) to improve X-ray interpretation. The system integrates several pre-trained vision models, including DenseNet architectures trained on large medical imaging datasets such as MIMIC and CheXpert, with advanced language models to deliver comprehensive and multilingual medical analysis. The proposed framework not only detects and classifies a wide range of medical conditions but also produces detailed clinical explanations in Arabic, helping to bridge the gap between automated image detection and clinical decision support. Based on feedback from practicing radiologists who reviewed the system’s capabilities, 95% indicated that the tool could significantly accelerate their X-ray interpretation workflow. Although the system has not yet undergone clinical validation, the strong positive response from medical experts suggests promising potential for future integration into clinical environments.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multimodal AI for Medical Imaging: Combining Vision Models with LLMs for Enhanced X-Ray Analysis

  • Hamza Salem,
  • Marko Pezer,
  • Muhammad Naveed Zafar,
  • Manuel Mazzara

摘要

This paper presents a novel multimodal approach to medical image analysis that combines state-of-the-art computer vision models with Large Language Models (LLMs) to improve X-ray interpretation. The system integrates several pre-trained vision models, including DenseNet architectures trained on large medical imaging datasets such as MIMIC and CheXpert, with advanced language models to deliver comprehensive and multilingual medical analysis. The proposed framework not only detects and classifies a wide range of medical conditions but also produces detailed clinical explanations in Arabic, helping to bridge the gap between automated image detection and clinical decision support. Based on feedback from practicing radiologists who reviewed the system’s capabilities, 95% indicated that the tool could significantly accelerate their X-ray interpretation workflow. Although the system has not yet undergone clinical validation, the strong positive response from medical experts suggests promising potential for future integration into clinical environments.