Scholarlink

Background and Aims <p>Patients increasingly use the internet and artificial intelligence chatbots to obtain health information, yet the readability, quality, understandability, and actionability of AI-generated gastrointestinal patient education remain unclear. This study compared gastrointestinal patient education from a professional society website with content generated by ChatGPT using validated health literacy instruments.</p> Methods <p>In this cross-sectional comparative study, 50 gastrointestinal patient education topics from the American Gastroenterological Association patient information website were paired with ChatGPT-generated responses using standardized prompts. Readability was assessed using the Flesch-Kincaid Grade Level, quality of treatment information was evaluated using the DISCERN instrument, and understandability and actionability were assessed using the Patient Education Materials Assessment Tool; scoring was performed by two blinded reviewers. Paired t tests were used to compare mean scores between sources, and intraclass correlation coefficients (ICCs) were used to assess interrater reliability between reviewers.</p> Results <p>Fifty paired topics were analyzed. The mean Flesch-Kincaid Grade Level was higher for ChatGPT than GI website materials (10.33 ± 1.5 vs 8.72 ± 1.7; mean difference, 1.61; P &lt; .001). Differences in DISCERN scores (63.5 ± 5.7 vs 64.3 ± 5.4; mean difference, − 0.8), PEMAT understandability (87.9% ± 6.9% vs 86.5% ± 7.8%; mean difference, 1.4%; P = .33), and PEMAT actionability (78.6% ± 9.8% vs 77.9% ± 10.2%; mean difference, 0.6%; P = .73) were not statistically significant. Inter-rater reliability was excellent across all measures, with intraclass correlation coefficients of 0.97 (95% CI, 0.95–0.99) for PEMAT understandability, 0.96 (95% CI, 0.94–0.98) for PEMAT actionability, and 0.99 (95% CI, 0.98–0.99) for DISCERN.</p> Conclusion <p>ChatGPT-generated gastrointestinal patient education demonstrated similar quality, understandability, and actionability compared with professional society materials but was written at a significantly higher reading level. Improving readability may enhance accessibility and support the safe integration of AI-generated patient education.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Readability, Quality, Understandability, and Actionability of ChatGPT Generated GI Patient Education Versus AGA Patient Center

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月2日

查看全文
  • Shivam Chandra,
  • Vineet Kumar,
  • Abhin Sapkota,
  • Robert Kwei-Nsoro,
  • Anas Almoghrabi

摘要

Background and Aims

Patients increasingly use the internet and artificial intelligence chatbots to obtain health information, yet the readability, quality, understandability, and actionability of AI-generated gastrointestinal patient education remain unclear. This study compared gastrointestinal patient education from a professional society website with content generated by ChatGPT using validated health literacy instruments.

Methods

In this cross-sectional comparative study, 50 gastrointestinal patient education topics from the American Gastroenterological Association patient information website were paired with ChatGPT-generated responses using standardized prompts. Readability was assessed using the Flesch-Kincaid Grade Level, quality of treatment information was evaluated using the DISCERN instrument, and understandability and actionability were assessed using the Patient Education Materials Assessment Tool; scoring was performed by two blinded reviewers. Paired t tests were used to compare mean scores between sources, and intraclass correlation coefficients (ICCs) were used to assess interrater reliability between reviewers.

Results

Fifty paired topics were analyzed. The mean Flesch-Kincaid Grade Level was higher for ChatGPT than GI website materials (10.33 ± 1.5 vs 8.72 ± 1.7; mean difference, 1.61; P < .001). Differences in DISCERN scores (63.5 ± 5.7 vs 64.3 ± 5.4; mean difference, − 0.8), PEMAT understandability (87.9% ± 6.9% vs 86.5% ± 7.8%; mean difference, 1.4%; P = .33), and PEMAT actionability (78.6% ± 9.8% vs 77.9% ± 10.2%; mean difference, 0.6%; P = .73) were not statistically significant. Inter-rater reliability was excellent across all measures, with intraclass correlation coefficients of 0.97 (95% CI, 0.95–0.99) for PEMAT understandability, 0.96 (95% CI, 0.94–0.98) for PEMAT actionability, and 0.99 (95% CI, 0.98–0.99) for DISCERN.

Conclusion

ChatGPT-generated gastrointestinal patient education demonstrated similar quality, understandability, and actionability compared with professional society materials but was written at a significantly higher reading level. Improving readability may enhance accessibility and support the safe integration of AI-generated patient education.