ChatGPT-4 is a widely used large language model that provides instant answers to a variety of health-related questions in different medical fields. This study aims to evaluate the reliability, quality, accuracy, and readability of ChatGPT’s responses to frequently asked questions regarding physical activity in children with cystic fibrosis (CF).The responses of ChatGPT-4 to 60 frequently asked questions related to physical activity and its effects on the condition of children with CF were categorized into five thematic (S1-S5) groups. These responses were then evaluated for reliability, quality, accuracy, and readability using the modified DISCERN (mDISCERN) tool, the Global Quality Scale (GQS), a five-point Likert scale, and the Flesch Reading Ease Scale (FRE), respectively.The mean scores for mDISCERN, GQS, and accuracy ranged from 3.38 (S2) to 3.82 (S4), 3.91 (S2, S4) to 4.25 (S5), and 4.27 (S1, S4) to 4.78 (S3), with overall means of 3.5, 3.98, and 4.38, respectively. The readability mean scores varied from 29.99 (S5) to 46.31 (S3), with a total mean of 38.07. The ICC values for the mDISCERN, GQS, and accuracy were 0.746, 0.666, and 0.665, respectively. Conclusion: This study revealed that ChatGPT-4 provides moderate to high levels of reliability, quality and accuracy in responses about physical activity in children with CF. Low FRE scores showed most responses were “difficult” for the target age group. Although ChatGPT-4 serves as a useful supplementary tool for patients with CF, professional supervision and further validation are essential for safe and effective use in clinical contexts. <Table Float="No" ID="Taba"> <tgroup cols="1"> <colspec align="left" colname="c1" colnum="1" /> <tbody> <row> <entry align="left" colname="c1"> What is Known: • Physical activity benefits children with cystic fibrosis (CF), yet access to reliable, understandable educational materials is limited. AI tools like ChatGPT-4 are increasingly used in health communication, but their reliability, accuracy, and readability remain uncertain. </entry> </row> <row> <entry align="left" colname="c1"> What is New: • This study systematically evaluates ChatGPT-4 responses to CF-related physical activity questions, showing moderate-to-high reliability, quality, and accuracy, but low readability, highlighting the need for adaptation for pediatric use. </entry> </row> </tbody> </tgroup> </Table>

Evaluation of ChatGPT-4 responses on physical activity guidance in children with cystic fibrosis: reliability, quality, and readability

期刊论文
发表日期：

2025年10月11日

查看全文

Zeliha Çelik,
Fulden Sari

摘要

ChatGPT-4 is a widely used large language model that provides instant answers to a variety of health-related questions in different medical fields. This study aims to evaluate the reliability, quality, accuracy, and readability of ChatGPT’s responses to frequently asked questions regarding physical activity in children with cystic fibrosis (CF).The responses of ChatGPT-4 to 60 frequently asked questions related to physical activity and its effects on the condition of children with CF were categorized into five thematic (S1-S5) groups. These responses were then evaluated for reliability, quality, accuracy, and readability using the modified DISCERN (mDISCERN) tool, the Global Quality Scale (GQS), a five-point Likert scale, and the Flesch Reading Ease Scale (FRE), respectively.The mean scores for mDISCERN, GQS, and accuracy ranged from 3.38 (S2) to 3.82 (S4), 3.91 (S2, S4) to 4.25 (S5), and 4.27 (S1, S4) to 4.78 (S3), with overall means of 3.5, 3.98, and 4.38, respectively. The readability mean scores varied from 29.99 (S5) to 46.31 (S3), with a total mean of 38.07. The ICC values for the mDISCERN, GQS, and accuracy were 0.746, 0.666, and 0.665, respectively.

Conclusion: This study revealed that ChatGPT-4 provides moderate to high levels of reliability, quality and accuracy in responses about physical activity in children with CF. Low FRE scores showed most responses were “difficult” for the target age group. Although ChatGPT-4 serves as a useful supplementary tool for patients with CF, professional supervision and further validation are essential for safe and effective use in clinical contexts.

What is Known:

• Physical activity benefits children with cystic fibrosis (CF), yet access to reliable, understandable educational materials is limited. AI tools like ChatGPT-4 are increasingly used in health communication, but their reliability, accuracy, and readability remain uncertain.

What is New:

• This study systematically evaluates ChatGPT-4 responses to CF-related physical activity questions, showing moderate-to-high reliability, quality, and accuracy, but low readability, highlighting the need for adaptation for pediatric use.