Scholarlink

Purpose <p>Knee osteoarthritis (OA) is common, and platelet‑rich plasma (PRP) is widely promoted despite inconsistent efficacy. As patients increasingly consult large language models (LLMs), concerns arise about the reliability, safety, and readability of AI‑generated content. This study compares patient‑facing PRP information produced by ChatGPT‑5.2, Gemini 3 Pro, and MediSearch AI using the Composite Clinical Reliability Score (CCRS).</p> Methods <p>Ninety PRP questions from Google’s <i>People Also Ask</i> were posed verbatim to all three LLMs. Two fellowship‑trained orthopaedic surgeons scored responses at two time points using CCRS, spanning factual accuracy, clinical correctness, guideline adherence, inverse hallucination severity, clinical safety errors, and evidence citation validity. Readability was assessed with Coleman–Liau Index (CLI). Inter‑rater reliability was assessed using intraclass correlation coefficients (ICC). Normality was tested with Shapiro–Wilk and inter‑model differences were evaluated using Friedman test.</p> Results <p>Across 270 responses, MediSearch AI achieved the highest CCRS (mean 76.8 ± 3.75), significantly outperforming ChatGPT‑5.2 (70.3 ± 5.28) and Gemini 3 Pro (58.2 ± 5.05) (<i>p &lt; </i>0.001). ICC for CCRS scoring demonstrated excellent agreement between reviewers (ICC = 0.87, 95%CI 0.82–0.91). MediSearch AI showed superior guideline adherence, hallucination control, safety framing, and citation validity. Readability differed significantly (<i>p &lt; </i>0.001): Gemini 3 Pro was most readable (mean CLI 13.2 ± 1.5), ChatGPT‑5.2 was moderately complex (13.3 ± 1.9), and MediSearch AI was least readable (15.4 ± 2.2).</p> Conclusion <p>LLMs differ markedly in PRP patient education. MediSearch AI delivered the most reliable but least readable content, Gemini 3 Pro produced the most accessible but least reliable outputs, and ChatGPT‑5.2 was intermediate. LLMs should serve only as adjuncts to professional consultation, as they cannot provide individualised medical advice, hence making balance between accuracy and accessibility critical.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Evaluating Large Language Models for Patient‑Facing Platelet‑Rich Plasma Information in Knee Osteoarthritis: Development and Application of the Composite Clinical Reliability Score (CCRS)

  • 期刊论文

  • 发表日期：

    2026年5月26日

查看全文
  • Sathish Muthu,
  • Dhibin Vikash Kolarpatti Ponnusamy,
  • Manimekalai Muthiah Rathinam,
  • Vibhu Krishnan Viswanathan,
  • Sathish Kumar Rajappan Chandra,
  • Khan Sharun

摘要

Purpose

Knee osteoarthritis (OA) is common, and platelet‑rich plasma (PRP) is widely promoted despite inconsistent efficacy. As patients increasingly consult large language models (LLMs), concerns arise about the reliability, safety, and readability of AI‑generated content. This study compares patient‑facing PRP information produced by ChatGPT‑5.2, Gemini 3 Pro, and MediSearch AI using the Composite Clinical Reliability Score (CCRS).

Methods

Ninety PRP questions from Google’s People Also Ask were posed verbatim to all three LLMs. Two fellowship‑trained orthopaedic surgeons scored responses at two time points using CCRS, spanning factual accuracy, clinical correctness, guideline adherence, inverse hallucination severity, clinical safety errors, and evidence citation validity. Readability was assessed with Coleman–Liau Index (CLI). Inter‑rater reliability was assessed using intraclass correlation coefficients (ICC). Normality was tested with Shapiro–Wilk and inter‑model differences were evaluated using Friedman test.

Results

Across 270 responses, MediSearch AI achieved the highest CCRS (mean 76.8 ± 3.75), significantly outperforming ChatGPT‑5.2 (70.3 ± 5.28) and Gemini 3 Pro (58.2 ± 5.05) (p < 0.001). ICC for CCRS scoring demonstrated excellent agreement between reviewers (ICC = 0.87, 95%CI 0.82–0.91). MediSearch AI showed superior guideline adherence, hallucination control, safety framing, and citation validity. Readability differed significantly (p < 0.001): Gemini 3 Pro was most readable (mean CLI 13.2 ± 1.5), ChatGPT‑5.2 was moderately complex (13.3 ± 1.9), and MediSearch AI was least readable (15.4 ± 2.2).

Conclusion

LLMs differ markedly in PRP patient education. MediSearch AI delivered the most reliable but least readable content, Gemini 3 Pro produced the most accessible but least reliable outputs, and ChatGPT‑5.2 was intermediate. LLMs should serve only as adjuncts to professional consultation, as they cannot provide individualised medical advice, hence making balance between accuracy and accessibility critical.