Scholarlink

Background <p>Deep learning-based anatomy segmentation holds promise for improving real-time guidance in complex surgeries such as robot-assisted minimally invasive esophagectomy (RAMIE). However, the clinical relevance of commonly used metrics for evaluating segmentation quality remains unclear, as previous assessments have lacked direct input from surgeons. This study aims to assess how well quantitative segmentation metrics reflect surgeons’ assessments of anatomical overlay accuracy and clinical usefulness during RAMIE.</p> Methods <p>We conducted a survey involving 26 upper gastrointestinal surgeons, including both trainee and attending surgeons, who assessed video clips of RAMIE procedures featuring deep learning-generated anatomical overlays. We correlated the surgeons’ qualitative evaluations of annotation accuracy and clinical usefulness with a comprehensive set of quantitative metrics, including overlap, distance, temporal, and error-specific measures. The analysis encompassed over 8000 manually annotated frames from 12 video clips, with overlays generated by two state-of-the-art deep learning models.</p> Results <p>Overlap and temporal consistency metrics show the strongest correlation with surgeon assessments. Distance-based and error-specific metrics correlate moderately. Novices show weaker correlations and tend to rate overlays more leniently. Qualitative feedback reveals issues like hallucinations and instability, often missed by current metrics.</p> Conclusion <p>Standard quantitative metrics partially reflect surgeon perceptions but should be complemented by surgeon-informed evaluations and task-specific metrics to better capture clinically relevant errors. Aligning metric design with surgical expertise is essential for the safe and effective integration of AI-guided anatomical segmentation in the operating room.</p> Graphical abstract <p></p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Do segmentation metrics reflect clinical reality? A surgeon-centered evaluation in robot-assisted minimally invasive esophagectomy

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年10月10日

查看全文
  • Ronald de Jong,
  • Yiping Li,
  • Romy van Jaarsveld,
  • Gino Kuiper,
  • Richard van Hillegersberg,
  • Jelle Ruurda,
  • Josien Pluim,
  • Marcel Breeuwer,
  • Yasmina Al Khalil

摘要

Background

Deep learning-based anatomy segmentation holds promise for improving real-time guidance in complex surgeries such as robot-assisted minimally invasive esophagectomy (RAMIE). However, the clinical relevance of commonly used metrics for evaluating segmentation quality remains unclear, as previous assessments have lacked direct input from surgeons. This study aims to assess how well quantitative segmentation metrics reflect surgeons’ assessments of anatomical overlay accuracy and clinical usefulness during RAMIE.

Methods

We conducted a survey involving 26 upper gastrointestinal surgeons, including both trainee and attending surgeons, who assessed video clips of RAMIE procedures featuring deep learning-generated anatomical overlays. We correlated the surgeons’ qualitative evaluations of annotation accuracy and clinical usefulness with a comprehensive set of quantitative metrics, including overlap, distance, temporal, and error-specific measures. The analysis encompassed over 8000 manually annotated frames from 12 video clips, with overlays generated by two state-of-the-art deep learning models.

Results

Overlap and temporal consistency metrics show the strongest correlation with surgeon assessments. Distance-based and error-specific metrics correlate moderately. Novices show weaker correlations and tend to rate overlays more leniently. Qualitative feedback reveals issues like hallucinations and instability, often missed by current metrics.

Conclusion

Standard quantitative metrics partially reflect surgeon perceptions but should be complemented by surgeon-informed evaluations and task-specific metrics to better capture clinically relevant errors. Aligning metric design with surgical expertise is essential for the safe and effective integration of AI-guided anatomical segmentation in the operating room.

Graphical abstract