Scholarlink

Objectives <p>This study evaluated the effect of enhancing a GPT-4 model with retrieval-augmented generation on its ability to diagnose and classify traumatic injuries based on radiology reports.</p> Materials and methods <p>In this prospective proof-of-concept study, we used retrieval-augmented generation as a zero-shot learning approach to provide expert knowledge from the RadioGraphics top ten reading list for trauma radiology to the GPT-4 model, creating the context-aware TraumaCB. Radiological report findings of 50 traumatic injuries were independently generated by two radiologists. The performance of the TraumaCB compared to the generic GPT-4 was evaluated by three board-certified radiologists, assessing the accuracy and trustworthiness of the chatbot responses in the 100 reports created.</p> Results <p>The TraumaCB achieved 100% correct diagnoses, 96% correct classification, and 87% correct grading, outperforming the generic GPT-4 with 93% correct diagnoses, 70% correct classification, and 48% correct grading. TraumaCB sources consistently achieved a median rating of 5.0 for explanation and trust. Challenges encountered mainly involved traumatic injuries lacking widely accepted classification systems.</p> Conclusion <p>Augmenting a commercial GPT-4 model with retrieval-augmented generation improves its diagnostic and classification capabilities, positioning it as a valuable tool for efficiently assessing traumatic injuries across various anatomical regions in trauma radiology.</p> Key Points <p><Emphasis Type="BoldItalic">Question</Emphasis> <i>Retrieval-augmented generation has the potential to enhance generic chatbots with task-specific knowledge of emergency radiology</i>.</p> <p><Emphasis Type="BoldItalic">Findings</Emphasis> <i>The TraumaCB excelled in accuracy, particularly in injury classification and grading, and provided explanations along with the sources used, increasing transparency and facilitating verification</i>.</p> <p><Emphasis Type="BoldItalic">Clinical relevance</Emphasis> <i>The TraumaCB provides accurate, fast, and transparent access to trauma radiology classifications, potentially increasing the efficiency of image interpretation in emergency departments and enabling customized reports based on local or individual preferences</i>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Retrieval-augmented generation improves precision and trust of a GPT-4 model for emergency radiology diagnosis and classification: a proof-of-concept study

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年2月14日

查看全文
  • Anna Fink,
  • Johanna Nattenmüller,
  • Stephan Rau,
  • Alexander Rau,
  • Hien Tran,
  • Fabian Bamberg,
  • Marco Reisert,
  • Elmar Kotter,
  • Thierno Diallo,
  • Maximilian F. Russe

摘要

Objectives

This study evaluated the effect of enhancing a GPT-4 model with retrieval-augmented generation on its ability to diagnose and classify traumatic injuries based on radiology reports.

Materials and methods

In this prospective proof-of-concept study, we used retrieval-augmented generation as a zero-shot learning approach to provide expert knowledge from the RadioGraphics top ten reading list for trauma radiology to the GPT-4 model, creating the context-aware TraumaCB. Radiological report findings of 50 traumatic injuries were independently generated by two radiologists. The performance of the TraumaCB compared to the generic GPT-4 was evaluated by three board-certified radiologists, assessing the accuracy and trustworthiness of the chatbot responses in the 100 reports created.

Results

The TraumaCB achieved 100% correct diagnoses, 96% correct classification, and 87% correct grading, outperforming the generic GPT-4 with 93% correct diagnoses, 70% correct classification, and 48% correct grading. TraumaCB sources consistently achieved a median rating of 5.0 for explanation and trust. Challenges encountered mainly involved traumatic injuries lacking widely accepted classification systems.

Conclusion

Augmenting a commercial GPT-4 model with retrieval-augmented generation improves its diagnostic and classification capabilities, positioning it as a valuable tool for efficiently assessing traumatic injuries across various anatomical regions in trauma radiology.

Key Points

Question Retrieval-augmented generation has the potential to enhance generic chatbots with task-specific knowledge of emergency radiology.

Findings The TraumaCB excelled in accuracy, particularly in injury classification and grading, and provided explanations along with the sources used, increasing transparency and facilitating verification.

Clinical relevance The TraumaCB provides accurate, fast, and transparent access to trauma radiology classifications, potentially increasing the efficiency of image interpretation in emergency departments and enabling customized reports based on local or individual preferences.