Background <p>Advancements in artificial intelligence have led to the widespread use of large language models such as ChatGPT in healthcare communication. Myotonic dystrophy type 1, a chronic and multisystemic neuromuscular disorder, poses significant challenges for patients in understanding disease progression, symptom management, and future planning. Given the scarcity of specialized resources for rare diseases, AI-based tools may offer valuable support by delivering accessible and relevant information. This study aimed to evaluate ChatGPT-4o’s performance in addressing patient-centered questions about myotonic dystrophy type 1.</p> Methods <p>This descriptive methodological study utilized ten frequently asked patient-style questions concerning myotonic dystrophy type 1, posed to ChatGPT-4o in Turkish using a standardized zero-shot prompting approach. A total of 53 specialist physicians in Physical Medicine and Rehabilitation and Neurology assessed the responses using a structured rubric across five domains: accuracy, currency, comprehensiveness, usefulness, and understandability. Each response was scored from 1 to 5.</p> Results <p>ChatGPT-4o achieved high specialist physicians’ satisfaction across all domains, with over 80% of responses rated ≥ 4. Accuracy received the highest score (85.7%), followed by usefulness and comprehensiveness (above 82%). Currency and understandability were rated slightly lower (81.9% and 81.5%, respectively).</p> Conclusion <p>ChatGPT-4o generated responses that were generally accurate, relevant, and understandable, demonstrating promise as an accessible resource for rare disease contexts such as myotonic dystrophy type 1. It also showed strong potential as a supportive informational tool for specialist physicians. Ongoing model refinement and integration with up-to-date clinical data are essential to optimize its performance and ensure its safe, equitable use in healthcare communication.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Evaluating the performance of ChatGPT in responding to myotonic dystrophy type 1 patient inquiries: a specialist physician-based study

  • Gülşah Çelik,
  • Hanife Hale Hekim,
  • Meral Bilgilisoy Filiz,
  • Naciye Füsun Toraman

摘要

Background

Advancements in artificial intelligence have led to the widespread use of large language models such as ChatGPT in healthcare communication. Myotonic dystrophy type 1, a chronic and multisystemic neuromuscular disorder, poses significant challenges for patients in understanding disease progression, symptom management, and future planning. Given the scarcity of specialized resources for rare diseases, AI-based tools may offer valuable support by delivering accessible and relevant information. This study aimed to evaluate ChatGPT-4o’s performance in addressing patient-centered questions about myotonic dystrophy type 1.

Methods

This descriptive methodological study utilized ten frequently asked patient-style questions concerning myotonic dystrophy type 1, posed to ChatGPT-4o in Turkish using a standardized zero-shot prompting approach. A total of 53 specialist physicians in Physical Medicine and Rehabilitation and Neurology assessed the responses using a structured rubric across five domains: accuracy, currency, comprehensiveness, usefulness, and understandability. Each response was scored from 1 to 5.

Results

ChatGPT-4o achieved high specialist physicians’ satisfaction across all domains, with over 80% of responses rated ≥ 4. Accuracy received the highest score (85.7%), followed by usefulness and comprehensiveness (above 82%). Currency and understandability were rated slightly lower (81.9% and 81.5%, respectively).

Conclusion

ChatGPT-4o generated responses that were generally accurate, relevant, and understandable, demonstrating promise as an accessible resource for rare disease contexts such as myotonic dystrophy type 1. It also showed strong potential as a supportive informational tool for specialist physicians. Ongoing model refinement and integration with up-to-date clinical data are essential to optimize its performance and ensure its safe, equitable use in healthcare communication.