Scholarlink

Abstract <p>Artificial intelligence (AI) and large language models (LLMs) are increasingly integrated into radiology, offering new possibilities for advanced imaging techniques, including cardiovascular magnetic resonance (CMR). This proof-of-concept study assessed four high-performing LLMs (Gemini 2.5 Pro, ChatGPT 4.1, DeepSeek V3, and Claude Opus 4) on their ability to generate CMR protocols for 140 hypothetical cardiac cases. AI-generated protocols were compared against a reference standard established by a consensus between two experienced cardiovascular radiologists, following the Society for Cardiovascular Magnetic Resonance (SCMR) recommendations. Descriptive statistics were used to quantify the concordance of LLM-generated sequences with the SCMR guidelines. Statistical agreement was measured using Cohen and Fleiss κ statistics. Gemini 2.5 Pro achieved the highest concordance, aligning with the SCMR guidelines in 71.5% of all evaluated scenarios. Overall, LLMs showed moderate agreement with the SCMR protocols, with Gemini 2.5 Pro again performing best (Cohen κ = 0.55). Agreement was substantial for mandatory CMR sequences (Fleiss κ ∈ [0.69, 0.74]) and predominantly fair for optional sequences. The tested LLMs demonstrate a potential to generate efficient and pathology-adapted CMR protocols. Under expert supervision, this capability could streamline the imaging workflow and help extend CMR to primary healthcare centers through protocol automation.</p> Relevance statement <p>The potential of Gemini 2.5 Pro, ChatGPT 4.1, DeepSeek V3, and Claude Opus 4 to suggest pathology-adapted CMR protocols could improve imaging throughput and help to expand access to advanced cardiac diagnostics in primary healthcare centers.</p> Key Points <p><UnorderedList Mark="Bullet"> <ItemContent> <p>The tested large language models show potential for generating CMR protocols.</p> </ItemContent> <ItemContent> <p>Substantial agreement on mandatory CMR sequences promises more efficient examinations.</p> </ItemContent> <ItemContent> <p>Automation of CMR protocols could help to improve access to this advanced technique outside major medical institutions.</p> </ItemContent> </UnorderedList></p> Graphical Abstract <p></p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Reliability of Gemini 2.5 Pro, ChatGPT 4.1, DeepSeek V3, and Claude Opus 4 in generating standardized CMR protocols

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年1月26日

查看全文
  • Răzvan-Andrei Licu,
  • Giuseppe Muscogiuri,
  • Davide Casartelli,
  • Anca Bacârea,
  • Marian Pop,
  • Andra-Maria Licu,
  • Daniele Sferratore,
  • Alessandro Caruso,
  • Marianna Mirchuk,
  • Piotr Tarkowski,
  • Jakub Byczkowski,
  • Sandro Sironi

摘要

Abstract

Artificial intelligence (AI) and large language models (LLMs) are increasingly integrated into radiology, offering new possibilities for advanced imaging techniques, including cardiovascular magnetic resonance (CMR). This proof-of-concept study assessed four high-performing LLMs (Gemini 2.5 Pro, ChatGPT 4.1, DeepSeek V3, and Claude Opus 4) on their ability to generate CMR protocols for 140 hypothetical cardiac cases. AI-generated protocols were compared against a reference standard established by a consensus between two experienced cardiovascular radiologists, following the Society for Cardiovascular Magnetic Resonance (SCMR) recommendations. Descriptive statistics were used to quantify the concordance of LLM-generated sequences with the SCMR guidelines. Statistical agreement was measured using Cohen and Fleiss κ statistics. Gemini 2.5 Pro achieved the highest concordance, aligning with the SCMR guidelines in 71.5% of all evaluated scenarios. Overall, LLMs showed moderate agreement with the SCMR protocols, with Gemini 2.5 Pro again performing best (Cohen κ = 0.55). Agreement was substantial for mandatory CMR sequences (Fleiss κ ∈ [0.69, 0.74]) and predominantly fair for optional sequences. The tested LLMs demonstrate a potential to generate efficient and pathology-adapted CMR protocols. Under expert supervision, this capability could streamline the imaging workflow and help extend CMR to primary healthcare centers through protocol automation.

Relevance statement

The potential of Gemini 2.5 Pro, ChatGPT 4.1, DeepSeek V3, and Claude Opus 4 to suggest pathology-adapted CMR protocols could improve imaging throughput and help to expand access to advanced cardiac diagnostics in primary healthcare centers.

Key Points

The tested large language models show potential for generating CMR protocols.

Substantial agreement on mandatory CMR sequences promises more efficient examinations.

Automation of CMR protocols could help to improve access to this advanced technique outside major medical institutions.

Graphical Abstract