Scholarlink

<p>Reproducibility in medico-legal personal injury assessment should be regarded as a fundamental prerequisite for transparent and equitable compensation. However, inter-rater agreement is rarely examined and quantified, even within a single regulatory framework. This study aimed to evaluate intra-system reproducibility in the Italian setting and to describe the behavior of a large language model (LLM). Using twenty case synopses derived from court-appointed expert reports in civil personal injury proceedings, we collected blinded assessments from fifteen human raters and from one LLM regarding percentage of permanent impairment and the degree of impairment-related suffering. Human raters were stratified by experience level (experts &gt; 20 years, specialists &lt; 10 years, final-year trainees). Inter-rater agreement among human assessors for biological damage was high (overall ICC 0.914; 95% CI 0.826–0.973), displaying an experience-related gradient. The LLM closely showed concordance with the court-appointed expert’s biological damage scores (bias + 0.05; Pearson’s r 0.95) but assigned substantially higher ratings of moral suffering than human assessors. Overall, biological damage evaluations showed high reproducibility, increasing with experience, whereas moral suffering exhibited a systematic divergence between human and LLM outputs.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Inter-rater reproducibility in medico-legal injury assessment: a pilot study with an exploratory comparison to a large language model

  • 期刊论文

  • 发表日期：

    2026年5月26日

查看全文
  • Alberto Blandino,
  • Federico Pacchioni,
  • Enrico Angelo Muccino,
  • Vittoradolfo Tambone,
  • Francesco De Micco,
  • Corinna Antonini,
  • Riccardo Zoja,
  • Guido Vittorio Travaini

摘要

Reproducibility in medico-legal personal injury assessment should be regarded as a fundamental prerequisite for transparent and equitable compensation. However, inter-rater agreement is rarely examined and quantified, even within a single regulatory framework. This study aimed to evaluate intra-system reproducibility in the Italian setting and to describe the behavior of a large language model (LLM). Using twenty case synopses derived from court-appointed expert reports in civil personal injury proceedings, we collected blinded assessments from fifteen human raters and from one LLM regarding percentage of permanent impairment and the degree of impairment-related suffering. Human raters were stratified by experience level (experts > 20 years, specialists < 10 years, final-year trainees). Inter-rater agreement among human assessors for biological damage was high (overall ICC 0.914; 95% CI 0.826–0.973), displaying an experience-related gradient. The LLM closely showed concordance with the court-appointed expert’s biological damage scores (bias + 0.05; Pearson’s r 0.95) but assigned substantially higher ratings of moral suffering than human assessors. Overall, biological damage evaluations showed high reproducibility, increasing with experience, whereas moral suffering exhibited a systematic divergence between human and LLM outputs.