Scholarlink

Background <p>Large language models are increasingly explored as clinical decision-support tools; however, their reliability in gynecologic practice remains insufficiently characterized. This study aimed to evaluate the quality of clinical management recommendations generated by ChatGPT-5 for human papillomavirus (HPV) infection and cervical premalignant lesions using a scenario-based expert assessment approach.</p> Methods <p>Fifteen clinically relevant scenarios reflecting common presentations of HPV infection and cervical premalignant lesions were developed in alignment with contemporary guideline frameworks. Each scenario was submitted to ChatGPT-5, and the resulting management and treatment recommendations were collected. The responses were independently evaluated by 50 experts, including 25 obstetrics and gynecology specialists and 25 gynecologic oncology specialists. Four predefined domains—guideline adherence, coverage, applicability, and overall satisfaction—were assessed using a 5-point Likert scale. Between-group comparisons, effect size analyses, and inter-rater reliability metrics were applied to explore differences in expert evaluations.</p> Results <p>The assessments revealed domain-specific variations between expert groups. Gynecologic oncologists assigned higher scores for guideline adherence across multiple scenarios, whereas obstetrics and gynecology specialists reported higher scores for coverage and overall satisfaction. Applicability ratings were consistently high in both groups, with minimal between-group differences. Inter-rater reliability varied across domains, suggesting differing levels of agreement depending on the evaluated aspect of the responses.</p> Conclusions <p>ChatGPT-5 demonstrated the capacity to generate clinically relevant recommendations for HPV-related cervical premalignant conditions. Nonetheless, variability in expert evaluations—particularly in relation to guideline adherence—highlights the need for specialist oversight and cautious integration of large language models into clinical gynecologic decision-making.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Evaluation of ChatGPT-5–generated clinical management recommendations for HPV infection and cervical premalignant lesions: a scenario-based expert assessment study

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年4月18日

查看全文
  • Mustafa Gokkaya,
  • Necim Yalcin,
  • Aysun Alci,
  • İbrahim Taşkum

摘要

Background

Large language models are increasingly explored as clinical decision-support tools; however, their reliability in gynecologic practice remains insufficiently characterized. This study aimed to evaluate the quality of clinical management recommendations generated by ChatGPT-5 for human papillomavirus (HPV) infection and cervical premalignant lesions using a scenario-based expert assessment approach.

Methods

Fifteen clinically relevant scenarios reflecting common presentations of HPV infection and cervical premalignant lesions were developed in alignment with contemporary guideline frameworks. Each scenario was submitted to ChatGPT-5, and the resulting management and treatment recommendations were collected. The responses were independently evaluated by 50 experts, including 25 obstetrics and gynecology specialists and 25 gynecologic oncology specialists. Four predefined domains—guideline adherence, coverage, applicability, and overall satisfaction—were assessed using a 5-point Likert scale. Between-group comparisons, effect size analyses, and inter-rater reliability metrics were applied to explore differences in expert evaluations.

Results

The assessments revealed domain-specific variations between expert groups. Gynecologic oncologists assigned higher scores for guideline adherence across multiple scenarios, whereas obstetrics and gynecology specialists reported higher scores for coverage and overall satisfaction. Applicability ratings were consistently high in both groups, with minimal between-group differences. Inter-rater reliability varied across domains, suggesting differing levels of agreement depending on the evaluated aspect of the responses.

Conclusions

ChatGPT-5 demonstrated the capacity to generate clinically relevant recommendations for HPV-related cervical premalignant conditions. Nonetheless, variability in expert evaluations—particularly in relation to guideline adherence—highlights the need for specialist oversight and cautious integration of large language models into clinical gynecologic decision-making.