Scholarlink

<p><b>Purpose:</b> A decision support pathway for general practitioners (GPs) was explored through automated referral letter analysis, with large language models’ (LLMs) diagnostic roles comprehensively evaluated. <b>Methods:</b> The in-context learning performance of ChatGPT and GPT-4 for diagnostic decision support was evaluated using referral letters. Synthetic referral letters generated by ChatGPT addressed data scarcity, with distributional congruence quantified via Kullback-Leibler divergence. Two fine-tuning frameworks were comparatively assessed: encoder-based pre-trained language models (PLMs) for diagnostic classification, and decoder-based LLMs adapted to multiple-choice question-answering paradigms. <b>Results:</b> GPT-4 showed suboptimal few-shot accuracy (0.544). Synthetic letters demonstrated high fidelity (KL-divergence&lt;0.05). Encoder-based PLMs consistently outperformed decoder-based LLMs when fine-tuned with augmented data, with BERT achieving 0.977 accuracy in mixed-train-collect-test protocols. Complementary F1 (0.9707) confirmed negligible diagnostic bias. <b>Conclusion:</b> LLMs exhibited insufficient diagnostic accuracy through both direct implementation (GPT-4 few-shot: 0.544) and fine-tuning approaches (accuracy 0.723), establishing fundamental limitations in clinical deployment. Crucially, their text-generation capability was leveraged for structured data augmentation, producing synthetic referral letters with high distributional fidelity (KL-divergence&lt;0.05). This validated methodology enabled superior diagnostic performance through encoder-based PLM fine-tuning, where BERT achieved near-clinical-utility accuracy (0.977) - demonstrating 25.4% relative improvement over best-performing LLMs. Implementation pathways consequently prioritize this hybrid framework: LLM-mediated data augmentation followed by resource-efficient PLM classifiers, currently undergoing neurologist-piloted validation before multicenter expansion.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Capability of large language models in assisting GPs with diagnoses

  • 期刊论文

  • 发表日期：

    2026年3月12日

查看全文
  • Ruibin Wang,
  • Abdul Rehman,
  • Tingting Li,
  • Rupert Page,
  • Hailing Li,
  • Xiaokun Wang,
  • Xiaosong Yang,
  • Jian Jun Zhang

摘要

Purpose: A decision support pathway for general practitioners (GPs) was explored through automated referral letter analysis, with large language models’ (LLMs) diagnostic roles comprehensively evaluated. Methods: The in-context learning performance of ChatGPT and GPT-4 for diagnostic decision support was evaluated using referral letters. Synthetic referral letters generated by ChatGPT addressed data scarcity, with distributional congruence quantified via Kullback-Leibler divergence. Two fine-tuning frameworks were comparatively assessed: encoder-based pre-trained language models (PLMs) for diagnostic classification, and decoder-based LLMs adapted to multiple-choice question-answering paradigms. Results: GPT-4 showed suboptimal few-shot accuracy (0.544). Synthetic letters demonstrated high fidelity (KL-divergence<0.05). Encoder-based PLMs consistently outperformed decoder-based LLMs when fine-tuned with augmented data, with BERT achieving 0.977 accuracy in mixed-train-collect-test protocols. Complementary F1 (0.9707) confirmed negligible diagnostic bias. Conclusion: LLMs exhibited insufficient diagnostic accuracy through both direct implementation (GPT-4 few-shot: 0.544) and fine-tuning approaches (accuracy 0.723), establishing fundamental limitations in clinical deployment. Crucially, their text-generation capability was leveraged for structured data augmentation, producing synthetic referral letters with high distributional fidelity (KL-divergence<0.05). This validated methodology enabled superior diagnostic performance through encoder-based PLM fine-tuning, where BERT achieved near-clinical-utility accuracy (0.977) - demonstrating 25.4% relative improvement over best-performing LLMs. Implementation pathways consequently prioritize this hybrid framework: LLM-mediated data augmentation followed by resource-efficient PLM classifiers, currently undergoing neurologist-piloted validation before multicenter expansion.