Scholarlink

Objective <p>Protocol deviations (PDs) are critical indicators of data quality and operational risk in clinical trials. Systematic under- or over-reporting of PDs at investigator sites can obscure true trial performance and compromise regulatory compliance. The goal of this paper is to present an integrated approach combining statistical outlier detection with generative AI-powered content analysis for detecting and characterizing outlier sites in PD reporting, enabling proactive risk management and targeted site oversight.</p> Results <p>In this study, the simaerep bootstrap algorithm was applied across 578 clinical studies encompassing 39,936 sites. It is shown that the method successfully identifies sites exhibiting statistically significant deviations from expected PD reporting rates, flagging both potential under-reporters and high-rate reporters. In addition, generative AI combined with hierarchical clustering automatically categorized diverse PD narratives into a validated taxonomy of 15 major categories and 47 specific topics through a "human-in-the-loop" refinement process with clinical operations experts. Independent validation of the GenAI topic classification using an "LLM-as-judge" approach achieved 88.7% accuracy (95% CI: 86.5%–90.6%). This integrated capability equips clinical operations teams to identify sites warranting further investigation and determine whether observed deviation patterns are study-systemic, country-specific, or site-specific, ultimately improving data integrity and risk-based audit strategy.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Protocol deviation outlier estimation combined with generative AI

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年4月10日

查看全文
  • Ioannis I. Spyroglou,
  • Magdaléna Řeháková,
  • Radim Klapka,
  • Anna Kacprzycka,
  • Maria Consuelo Luguera Fernandez,
  • Ronojit Adyanthaya,
  • Kyle Young,
  • Jolie Weintraub

摘要

Objective

Protocol deviations (PDs) are critical indicators of data quality and operational risk in clinical trials. Systematic under- or over-reporting of PDs at investigator sites can obscure true trial performance and compromise regulatory compliance. The goal of this paper is to present an integrated approach combining statistical outlier detection with generative AI-powered content analysis for detecting and characterizing outlier sites in PD reporting, enabling proactive risk management and targeted site oversight.

Results

In this study, the simaerep bootstrap algorithm was applied across 578 clinical studies encompassing 39,936 sites. It is shown that the method successfully identifies sites exhibiting statistically significant deviations from expected PD reporting rates, flagging both potential under-reporters and high-rate reporters. In addition, generative AI combined with hierarchical clustering automatically categorized diverse PD narratives into a validated taxonomy of 15 major categories and 47 specific topics through a "human-in-the-loop" refinement process with clinical operations experts. Independent validation of the GenAI topic classification using an "LLM-as-judge" approach achieved 88.7% accuracy (95% CI: 86.5%–90.6%). This integrated capability equips clinical operations teams to identify sites warranting further investigation and determine whether observed deviation patterns are study-systemic, country-specific, or site-specific, ultimately improving data integrity and risk-based audit strategy.