Purpose <p>To assess narrative clinical documentation quality in assisted reproductive technology (ART) consultations, by applying a transparent, reproducible AI-assisted scoring framework.</p> Methods <p>In this cross-sectional observational study, 90 consecutive initial ART consultation records from 9 clinicians were evaluated using a predefined, domain-based documentation scoring framework. Documentation quality was assessed across three domains: Regulatory Alignment, Clarity of Plan of Action, and Disclosure Readiness. A rule-constrained AI-assisted tool applied fixed scoring criteria. Domain scores were normalised, and mixed-effects models were used to examine domain differences, associations, and clinician-level effects. Human–AI concordance was assessed in a blinded random subset scored by an independent expert clinician. Repeatability was assessed through repeated rescoring under identical conditions.</p> Results <p>Mean normalised documentation scores differed significantly across domains (Regulatory Alignment 0.830, Clarity of Plan of Action 0.677, Disclosure Readiness 0.753, Wald <i>χ</i><sup>2</sup> = 284.43 <i>p</i> &lt; 0.001). Domain ordering was consistent across documents, with lowest scoring for Plan Clarity (Kendall’s W = 0.53, <i>p</i> &lt; 0.001), and was unchanged after adjustment for consultation type and documentation length. Score agreement between AI and the expert was substantial (weighted Kappa = 0.65), with high concordance in domain ordering (mean Kendall T = 0.84) and no significant differences overall (<i>p</i> = 0.10).</p> Conclusions <p>A constrained, domain-based AI scoring framework enables transparent and reproducible quantification of narrative documentation quality in reproductive medicine. This approach may help generate formative feedback and hypotheses for targeted documentation improvement, which should be evaluated against downstream outcomes in future work.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

AI-assisted assessment of narrative documentation quality in ART consultations using a domain-based framework

  • Athanasios Papathanasiou

摘要

Purpose

To assess narrative clinical documentation quality in assisted reproductive technology (ART) consultations, by applying a transparent, reproducible AI-assisted scoring framework.

Methods

In this cross-sectional observational study, 90 consecutive initial ART consultation records from 9 clinicians were evaluated using a predefined, domain-based documentation scoring framework. Documentation quality was assessed across three domains: Regulatory Alignment, Clarity of Plan of Action, and Disclosure Readiness. A rule-constrained AI-assisted tool applied fixed scoring criteria. Domain scores were normalised, and mixed-effects models were used to examine domain differences, associations, and clinician-level effects. Human–AI concordance was assessed in a blinded random subset scored by an independent expert clinician. Repeatability was assessed through repeated rescoring under identical conditions.

Results

Mean normalised documentation scores differed significantly across domains (Regulatory Alignment 0.830, Clarity of Plan of Action 0.677, Disclosure Readiness 0.753, Wald χ2 = 284.43 p < 0.001). Domain ordering was consistent across documents, with lowest scoring for Plan Clarity (Kendall’s W = 0.53, p < 0.001), and was unchanged after adjustment for consultation type and documentation length. Score agreement between AI and the expert was substantial (weighted Kappa = 0.65), with high concordance in domain ordering (mean Kendall T = 0.84) and no significant differences overall (p = 0.10).

Conclusions

A constrained, domain-based AI scoring framework enables transparent and reproducible quantification of narrative documentation quality in reproductive medicine. This approach may help generate formative feedback and hypotheses for targeted documentation improvement, which should be evaluated against downstream outcomes in future work.