Scholarlink

Background <p>To evaluate the performance of two AI systems, ChatGPT 4.0 and Algor, in generating concept maps from validated otolaryngology clinical practice guidelines.</p> Methods <p>Concept maps were generated by ChatGPT 4.0 and Algor from four American Academy of Otolaryngology-Head and Neck Surgery Foundation (AAO-HNSF) clinical practice guidelines. Eight otolaryngology specialists evaluated the generated concept maps using the AI-Map questionnaire, covering concept identification, relationship establishment, hierarchical structure representation, and visual presentation. Chi-square tests and Kendall’s tau coefficient were used for statistical analysis.</p> Results <p>While no consistent superiority was observed across all guidelines, both AI systems demonstrated unique strengths. ChatGPT excelled in representing cross-connections between concepts and layout optimization, particularly for the Rhinoplasty guidelines (χ²=6.000, <i>p</i> = 0.050 for cross-connections). Algor showed strengths in capturing main themes and distinguishing general/abstract concepts, especially in the BPVV and Tympanostomy Tube guidelines (χ²=8.000, <i>p</i> = 0.046 for main themes in BPVV). Statistically significant differences were found in representing dynamic nature (favouring H&amp;NMass-GPT, χ²=7.571, <i>p</i> = 0.023) and overall value and usefulness (favouring H&amp;NMass-Algor, χ²=7.905, <i>p</i> = 0.019) for the H&amp;N Masses guidelines.</p> Conclusion <p>AI systems showed potential in automating concept map creation from otolaryngology guidelines, with performance varying across different medical topics and evaluation criteria. Further research is required to optimize AI systems for medical education and knowledge representation, highlighting their promise and current limitations.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

ChatGPT 4.0 and algor in generating concept maps: an observational study

  • 期刊论文

  • 发表日期：

    2025年2月20日

查看全文
  • Antonino Maniaci,
  • Caterina Gagliano,
  • Valerio Salerno,
  • Nicole Cilia,
  • Salvatore Lavalle,
  • Alberto Maria Saibene,
  • Giovanni Cammaroto,
  • Carlos Chiesa-Estomba,
  • Thomas Radulesco,
  • Luigi Vaira,
  • Giannicola Iannella,
  • Nicolas Fakhry,
  • Jerome Rene Lechien

摘要

Background

To evaluate the performance of two AI systems, ChatGPT 4.0 and Algor, in generating concept maps from validated otolaryngology clinical practice guidelines.

Methods

Concept maps were generated by ChatGPT 4.0 and Algor from four American Academy of Otolaryngology-Head and Neck Surgery Foundation (AAO-HNSF) clinical practice guidelines. Eight otolaryngology specialists evaluated the generated concept maps using the AI-Map questionnaire, covering concept identification, relationship establishment, hierarchical structure representation, and visual presentation. Chi-square tests and Kendall’s tau coefficient were used for statistical analysis.

Results

While no consistent superiority was observed across all guidelines, both AI systems demonstrated unique strengths. ChatGPT excelled in representing cross-connections between concepts and layout optimization, particularly for the Rhinoplasty guidelines (χ²=6.000, p = 0.050 for cross-connections). Algor showed strengths in capturing main themes and distinguishing general/abstract concepts, especially in the BPVV and Tympanostomy Tube guidelines (χ²=8.000, p = 0.046 for main themes in BPVV). Statistically significant differences were found in representing dynamic nature (favouring H&NMass-GPT, χ²=7.571, p = 0.023) and overall value and usefulness (favouring H&NMass-Algor, χ²=7.905, p = 0.019) for the H&N Masses guidelines.

Conclusion

AI systems showed potential in automating concept map creation from otolaryngology guidelines, with performance varying across different medical topics and evaluation criteria. Further research is required to optimize AI systems for medical education and knowledge representation, highlighting their promise and current limitations.