Scholarlink

Background <p>Delayed diagnosis of rare orbital diseases is attributed to limited clinical awareness. Building on prior evidence of multimodal large language models (MLLMs) for detecting common ocular conditions, this study aims to evaluate whether integrating multimodal clinical data can enhance the diagnostic accuracy of MLLMs for rare orbital diseases.</p> Methods <p>We conducted a multinational, multiracial, retrospective study. Two datasets were analyzed: Dataset 1, containing 6,786 single-eye photographs from China, was used to fine-tune a contrastive language-image pre-training (CLIP) for preliminary classification of healthy eyes, orbital diseases, and non-orbital diseases, and to compare its performance against three traditional models and three next-generation models. Dataset 2, comprising 170 participants from China, Singapore, and Thailand, was used to evaluate a MLLM (GPT-4o-Latest). Sequential sensitivity analysis assessed the impact of adding external eye photographs, chief complaints, racial information, and diagnostic reasoning prompts. An AI agent combining the CLIP model with GPT-4o-Latest was further evaluated. The model’s ability to generate medical reports and examination recommendations was also assessed.</p> Results <p>Here we show that the CLIP model achieves 90.21% preliminary detection accuracy, surpassing all baseline models. MLLM detection accuracy improves significantly with the inclusion of multimodal inputs. When relying on external eye images, the top-5 accuracy is 25.68%. The combined agent raises top-5 accuracy to 85.29%. Generated reports and recommendations display high accuracy, readability, completeness, and low potential for harm.</p> Conclusions <p>Our study demonstrates the potential of MLLM in improving diagnostic accuracy and supporting clinical decision-making for rare orbital diseases.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Sequential sensitivity analysis of multimodal large language models for rare orbital disease detection

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年2月20日

查看全文
  • Chaoyu Lei,
  • Kaiyuan Ji,
  • Chen Zhao,
  • Sisi Zhong,
  • Chenyu Cao,
  • Hao Chen,
  • Chee Chew Yip,
  • Sunisa Sintuwong,
  • Jianbin Ding,
  • P. S. Pandiyan,
  • Sunsern Wattanaphanich,
  • Luke Johnston,
  • Yujie Ren,
  • Xuran Duan,
  • Guangtao Zhai,
  • Huifang Zhou

摘要

Background

Delayed diagnosis of rare orbital diseases is attributed to limited clinical awareness. Building on prior evidence of multimodal large language models (MLLMs) for detecting common ocular conditions, this study aims to evaluate whether integrating multimodal clinical data can enhance the diagnostic accuracy of MLLMs for rare orbital diseases.

Methods

We conducted a multinational, multiracial, retrospective study. Two datasets were analyzed: Dataset 1, containing 6,786 single-eye photographs from China, was used to fine-tune a contrastive language-image pre-training (CLIP) for preliminary classification of healthy eyes, orbital diseases, and non-orbital diseases, and to compare its performance against three traditional models and three next-generation models. Dataset 2, comprising 170 participants from China, Singapore, and Thailand, was used to evaluate a MLLM (GPT-4o-Latest). Sequential sensitivity analysis assessed the impact of adding external eye photographs, chief complaints, racial information, and diagnostic reasoning prompts. An AI agent combining the CLIP model with GPT-4o-Latest was further evaluated. The model’s ability to generate medical reports and examination recommendations was also assessed.

Results

Here we show that the CLIP model achieves 90.21% preliminary detection accuracy, surpassing all baseline models. MLLM detection accuracy improves significantly with the inclusion of multimodal inputs. When relying on external eye images, the top-5 accuracy is 25.68%. The combined agent raises top-5 accuracy to 85.29%. Generated reports and recommendations display high accuracy, readability, completeness, and low potential for harm.

Conclusions

Our study demonstrates the potential of MLLM in improving diagnostic accuracy and supporting clinical decision-making for rare orbital diseases.