Scholarlink

<p>Generative AI is rapidly entering patient education workflows, yet its safety profile for concussion management remains undefined. Utilizing the CHART framework, this cross-sectional audit assessed five platforms, specifically isolating retrieval-augmented generation (RAG) architectures against standard pre-trained Large Language Models (LLMs). We extracted 11 high-volume patient queries from Google Trends and administered them via a zero-shot protocol. Two blinded neurosurgeons then scored the outputs against the 2023 Amsterdam Consensus Statement using four validated instruments: DISCERN and EQIP to evaluate treatment and information quality, GQS for global content quality, and JAMA benchmarks for transparency. Reliability metrics diverged significantly across models (DISCERN and EQIP, <i>p</i> &lt; 0.001). Perplexity Pro secured the highest DISCERN (47.36 ± 4.84) and EQIP (65.00 ± 5.48) values, statistically surpassing foundational models like ChatGPT and Gemini (<i>p</i> &lt; 0.01) — a performance gap likely driven by its RAG design. In contrast, GQS scores did not differ significantly across models (<i>p</i> = 0.373), and JAMA-based transparency remained uniformly low (<i>p</i> &lt; 0.001). Readability was assessed using six standard indices (FRES, FKGL, GFI, CLI, ARI, and SMOG), revealing that all models exceeded the 6th-grade reading level; most surpassed 10th-grade, with Perplexity Pro lowest at FKGL = 7.46. Although retrieval-augmented systems improve clinical accuracy, current iterations fail to provide transparent or readable advice. Clinical integration therefore requires rigorous human-in-the-loop verification and a shift toward plain-language algorithm optimization.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Reliability and readability of five AI chatbots for concussion health advice across retrieval augmented and pretrained models

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月3日

查看全文
  • Hefang Huang,
  • Caihong Zhang,
  • Hao Hu,
  • Xiang Tong

摘要

Generative AI is rapidly entering patient education workflows, yet its safety profile for concussion management remains undefined. Utilizing the CHART framework, this cross-sectional audit assessed five platforms, specifically isolating retrieval-augmented generation (RAG) architectures against standard pre-trained Large Language Models (LLMs). We extracted 11 high-volume patient queries from Google Trends and administered them via a zero-shot protocol. Two blinded neurosurgeons then scored the outputs against the 2023 Amsterdam Consensus Statement using four validated instruments: DISCERN and EQIP to evaluate treatment and information quality, GQS for global content quality, and JAMA benchmarks for transparency. Reliability metrics diverged significantly across models (DISCERN and EQIP, p < 0.001). Perplexity Pro secured the highest DISCERN (47.36 ± 4.84) and EQIP (65.00 ± 5.48) values, statistically surpassing foundational models like ChatGPT and Gemini (p < 0.01) — a performance gap likely driven by its RAG design. In contrast, GQS scores did not differ significantly across models (p = 0.373), and JAMA-based transparency remained uniformly low (p < 0.001). Readability was assessed using six standard indices (FRES, FKGL, GFI, CLI, ARI, and SMOG), revealing that all models exceeded the 6th-grade reading level; most surpassed 10th-grade, with Perplexity Pro lowest at FKGL = 7.46. Although retrieval-augmented systems improve clinical accuracy, current iterations fail to provide transparent or readable advice. Clinical integration therefore requires rigorous human-in-the-loop verification and a shift toward plain-language algorithm optimization.