Scholarlink

Automating multimodal medical imaging is vital due to the complexity of integrating diagnostic processes and patient interfaces into cohesive clinical workflows. This paper presents MedXpert-CAD, a multimodal multi-agent system (MMAS) that employs large language models (LLMs) and vision encoders for an evidence-based VQA chatbot, supporting multi-task diagnosis for X-ray respiratory diseases and MRI lumbar spinal stenosis (LSS). MedXpert-CAD features four dedicated sub-agents: the supervisor agent, which queries users and directs them to modality-specific tools like X-ray respiratory disease analysis or LSS prediction; the online search agent, which retrieves data from PubMed or Google Search when needed; the X-ray expert agent, which processes images to provide multi-label classification, bounding box detection via saliency mapping, VQA responses, and structured reports; and the LSS expert agent, which handles MRI or DICOM files, offering sagittal and axial view segmentation, spinal pathological measurements, disk herniation diagnosis, and structured reports. MedXpert-CAD integrates the Model Context Protocol (MCP) with specialized agents to overcome AI model limitations, ensuring interoperable architecture and evidence-based diagnostics. The evaluation used the DeepEval framework and LLM-as-a-judge methodology, assessing task completion (TC), contextual relevance (CR), and tool accuracy (TN). GPT-4o showed exceptional performance with TC rates of 90% (X-ray) and 96% (MRI), and 100% TN in the single-agent framework, compared to Llama-3.2's 61% TC (X-ray) and 63% TC (MRI) with 100% TN. In the multi-agent framework, GPT-4o achieved 80% TC and 87% TN, surpassing Llama-3.2's 65% TC and 47% TN, with contextual relevance peaking at 83%. Segmentation efficacy peaked on sagittal views with a Dice Similarity Coefficient (DSC) of 96.43% and 99.66% accuracy, outperforming axial views (DSC 92.71%, accuracy 99.88%). Report generation metrics highlighted GPT-4o's superiority with METEOR 29%, ROUGE-L 19%, FrugalScore 57%, Blue-1 31%, and BERTScore-F1 84% for chest X-ray, and METEOR 18%, ROUGE-L 11%, FrugalScore 59%, Blue-1 11%, and BERTScore-F1 82% for LSS. This system enhances patient care and empowers healthcare professionals with advanced imaging tools.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

MedXpert-CAD: A Multimodal Multi-agentic System for Clinical Imaging Analysis via Model Context Protocol LLM-Driven Agentic Workflows

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Mukhlis Raza,
  • Saied Salem,
  • Afnan Habib,
  • Osamah Abdulmahmod,
  • Hyunwook Kwon,
  • Jamil Hussain,
  • Mugahed A. Al-antari

摘要

Automating multimodal medical imaging is vital due to the complexity of integrating diagnostic processes and patient interfaces into cohesive clinical workflows. This paper presents MedXpert-CAD, a multimodal multi-agent system (MMAS) that employs large language models (LLMs) and vision encoders for an evidence-based VQA chatbot, supporting multi-task diagnosis for X-ray respiratory diseases and MRI lumbar spinal stenosis (LSS). MedXpert-CAD features four dedicated sub-agents: the supervisor agent, which queries users and directs them to modality-specific tools like X-ray respiratory disease analysis or LSS prediction; the online search agent, which retrieves data from PubMed or Google Search when needed; the X-ray expert agent, which processes images to provide multi-label classification, bounding box detection via saliency mapping, VQA responses, and structured reports; and the LSS expert agent, which handles MRI or DICOM files, offering sagittal and axial view segmentation, spinal pathological measurements, disk herniation diagnosis, and structured reports. MedXpert-CAD integrates the Model Context Protocol (MCP) with specialized agents to overcome AI model limitations, ensuring interoperable architecture and evidence-based diagnostics. The evaluation used the DeepEval framework and LLM-as-a-judge methodology, assessing task completion (TC), contextual relevance (CR), and tool accuracy (TN). GPT-4o showed exceptional performance with TC rates of 90% (X-ray) and 96% (MRI), and 100% TN in the single-agent framework, compared to Llama-3.2's 61% TC (X-ray) and 63% TC (MRI) with 100% TN. In the multi-agent framework, GPT-4o achieved 80% TC and 87% TN, surpassing Llama-3.2's 65% TC and 47% TN, with contextual relevance peaking at 83%. Segmentation efficacy peaked on sagittal views with a Dice Similarity Coefficient (DSC) of 96.43% and 99.66% accuracy, outperforming axial views (DSC 92.71%, accuracy 99.88%). Report generation metrics highlighted GPT-4o's superiority with METEOR 29%, ROUGE-L 19%, FrugalScore 57%, Blue-1 31%, and BERTScore-F1 84% for chest X-ray, and METEOR 18%, ROUGE-L 11%, FrugalScore 59%, Blue-1 11%, and BERTScore-F1 82% for LSS. This system enhances patient care and empowers healthcare professionals with advanced imaging tools.