Scholarlink

Background <p>Accurate differential diagnosis (DDx) in neuroradiology is challenging and remains vulnerable to cognitive biases and inter-reader variability in clinical practice. Despite expert training, human-generated DDx may be limited in breadth. Large language models (LLMs) have shown promise in assisting DDx generation; however, their performance may be limited by incomplete domain knowledge and concerns regarding data privacy and deployment latency. Retrieval-augmented generation (RAG) offers a potential solution by integrating external knowledge bases to enhance diagnostic accuracy while enabling on-premises implementation.</p> Objectives <p>To determine whether RAG improves LLM performance for neuroradiologic DDx and to compare proprietary versus open-source systems.</p> Methods <p>We assembled a dataset of 737 challenging neuroradiology cases. Each case provided free-text imaging findings, clinical information, and a ground-truth diagnosis. A 1195-entry brain DDx knowledge base was synthesized using an LLM. Seven LLMs (five open-source, two proprietary) each generated ten-item DDx lists with and without RAG.</p> Results <p>Across models, RAG significantly improved DDx accuracy for the majority of LLMs (<i>p</i> &lt; 0.05). Among proprietary systems, O3 + RAG achieved 72.9% (Top-1), 85.8% (Top-3), 88.7% (Top-5), and 92.3% (Top-10). Among open-source options, Gemma-3 + RAG provided a strong accuracy–efficiency–privacy balance at 49.4% (Top-1), 71.4% (Top-3), 79.2% (Top-5), and 87.1% (Top-10).</p> Conclusion <p>Coupling LLMs to a curated neuroradiology knowledge base via RAG markedly increases diagnostic accuracy and reduces the performance gap between open-source and proprietary models, while maintaining rapid, privacy-preserving inference suitable for clinical integration.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Toward on-premises RAG pipelines for AI-assisted neuroradiology: practical considerations on privacy, accuracy, and timeliness

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月31日

查看全文
  • Yu-Yun Chang,
  • Teng-Yi Huang,
  • Yi-Ju Pan,
  • Yu-Heng Liu,
  • Kuei-Hong Kuo

摘要

Background

Accurate differential diagnosis (DDx) in neuroradiology is challenging and remains vulnerable to cognitive biases and inter-reader variability in clinical practice. Despite expert training, human-generated DDx may be limited in breadth. Large language models (LLMs) have shown promise in assisting DDx generation; however, their performance may be limited by incomplete domain knowledge and concerns regarding data privacy and deployment latency. Retrieval-augmented generation (RAG) offers a potential solution by integrating external knowledge bases to enhance diagnostic accuracy while enabling on-premises implementation.

Objectives

To determine whether RAG improves LLM performance for neuroradiologic DDx and to compare proprietary versus open-source systems.

Methods

We assembled a dataset of 737 challenging neuroradiology cases. Each case provided free-text imaging findings, clinical information, and a ground-truth diagnosis. A 1195-entry brain DDx knowledge base was synthesized using an LLM. Seven LLMs (five open-source, two proprietary) each generated ten-item DDx lists with and without RAG.

Results

Across models, RAG significantly improved DDx accuracy for the majority of LLMs (p < 0.05). Among proprietary systems, O3 + RAG achieved 72.9% (Top-1), 85.8% (Top-3), 88.7% (Top-5), and 92.3% (Top-10). Among open-source options, Gemma-3 + RAG provided a strong accuracy–efficiency–privacy balance at 49.4% (Top-1), 71.4% (Top-3), 79.2% (Top-5), and 87.1% (Top-10).

Conclusion

Coupling LLMs to a curated neuroradiology knowledge base via RAG markedly increases diagnostic accuracy and reduces the performance gap between open-source and proprietary models, while maintaining rapid, privacy-preserving inference suitable for clinical integration.