Scholarlink

Purpose <p>Obstructive Sleep Apnea (OSA) remains underdiagnosed despite its high prevalence and health risks. With increasing patient reliance on artificial intelligence (AI) for health guidance, this study aimed to evaluate the accuracy, comprehensiveness, and empathy of responses generated by large language models (LLMs) to OSA-related queries, as judged by orthodontists.</p> Methods <p>Ten commonly asked OSA-related patient questions were posed to five LLMs (ChatGPT-3.4, ChatGPT-4.0, LLaMA, Bing, and Gemini). Responses were collected and anonymized, then evaluated by 20 orthodontists with over ten years of clinical experience. Ratings were assigned using a standardized 4-point scale (accuracy, comprehensiveness, empathy, and clinical relevance). Statistical analysis was conducted using the Kruskal–Wallis test with Bonferroni post hoc correction.</p> Results <p>Significant differences were observed across models (<i>p</i> &lt; 0.05). LLaMA consistently outperformed other models, offering comprehensive, empathetic, and clinically relevant responses. Bing was valued for citation-based outputs, while Gemini was appreciated for its disclaimers. ChatGPT models performed less favourably across multiple domains.</p> Conclusion <p>LLaMA demonstrated superior response quality in OSA-related interactions; however, AI tools should be considered complementary resources rather than substitutes for professional diagnosis and care. Integration with clinician oversight remains essential for patient safety.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Assessing Large Language Models for Patient Interaction in Orthodontics: A Brief Evaluation on Obstructive Sleep Apnea Queries

  • 期刊论文

  • 发表日期：

    2025年11月13日

查看全文
  • Tasneem Naeem Ahmed,
  • Monis Raza,
  • Achint Juneja,
  • Shraddha Bhalla,
  • Prachi Mehra,
  • Vikas Thakur

摘要

Purpose

Obstructive Sleep Apnea (OSA) remains underdiagnosed despite its high prevalence and health risks. With increasing patient reliance on artificial intelligence (AI) for health guidance, this study aimed to evaluate the accuracy, comprehensiveness, and empathy of responses generated by large language models (LLMs) to OSA-related queries, as judged by orthodontists.

Methods

Ten commonly asked OSA-related patient questions were posed to five LLMs (ChatGPT-3.4, ChatGPT-4.0, LLaMA, Bing, and Gemini). Responses were collected and anonymized, then evaluated by 20 orthodontists with over ten years of clinical experience. Ratings were assigned using a standardized 4-point scale (accuracy, comprehensiveness, empathy, and clinical relevance). Statistical analysis was conducted using the Kruskal–Wallis test with Bonferroni post hoc correction.

Results

Significant differences were observed across models (p < 0.05). LLaMA consistently outperformed other models, offering comprehensive, empathetic, and clinically relevant responses. Bing was valued for citation-based outputs, while Gemini was appreciated for its disclaimers. ChatGPT models performed less favourably across multiple domains.

Conclusion

LLaMA demonstrated superior response quality in OSA-related interactions; however, AI tools should be considered complementary resources rather than substitutes for professional diagnosis and care. Integration with clinician oversight remains essential for patient safety.