Scholarlink

Background <p>Large language models such as ChatGPT are increasingly used in biomedical writing; however, their ability to generate accurate and verifiable references in oral and maxillofacial surgery (OMS) remains uncertain.</p> Methods <p>This cross-sectional observational study evaluated references generated by ChatGPT-4 in response to ten standardized OMS prompts. A total of 500 Vancouver-style references were verified against PubMed, CrossRef, Google Scholar, and publisher databases and classified as accurate, partially correct, or fabricated.</p> Results <p>Of the 500 references, 203 (40.6%) were accurate, 165 (33.0%) were partially correct, and 132 (26.4%) were fabricated. Accuracy varied across subspecialties, with the highest accuracy observed in maxillofacial trauma and the lowest in oral pathology and oncology. Inter-reviewer agreement was excellent (κ = 0.87).</p> Conclusion <p>ChatGPT-4 demonstrates inconsistent citation accuracy in OMS. AI-generated references should be independently verified prior to scholarly use to maintain academic integrity.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Evaluating the accuracy of ChatGPT-4 generated references in oral and maxillofacial surgery: a preliminary observational study

  • 期刊论文

  • 发表日期：

    2026年3月21日

查看全文
  • Anuj Jain,
  • Yash Merchant

摘要

Background

Large language models such as ChatGPT are increasingly used in biomedical writing; however, their ability to generate accurate and verifiable references in oral and maxillofacial surgery (OMS) remains uncertain.

Methods

This cross-sectional observational study evaluated references generated by ChatGPT-4 in response to ten standardized OMS prompts. A total of 500 Vancouver-style references were verified against PubMed, CrossRef, Google Scholar, and publisher databases and classified as accurate, partially correct, or fabricated.

Results

Of the 500 references, 203 (40.6%) were accurate, 165 (33.0%) were partially correct, and 132 (26.4%) were fabricated. Accuracy varied across subspecialties, with the highest accuracy observed in maxillofacial trauma and the lowest in oral pathology and oncology. Inter-reviewer agreement was excellent (κ = 0.87).

Conclusion

ChatGPT-4 demonstrates inconsistent citation accuracy in OMS. AI-generated references should be independently verified prior to scholarly use to maintain academic integrity.