Purpose: Surgical video review is essential for minimally invasive surgical training, but manual annotation of surgical steps is time-consuming and limits scalability. We propose a weakly supervised pre-training framework that leverages unannotated or heterogeneously labeled surgical videos to improve automated surgical step recognition. Methods: We evaluate three types of weak labels derived from unannotated datasets: (1) surgical phases from the same or other procedures, (2) surgical steps from different procedure types, and (3) intraoperative time progression. Using datasets from four robotic-assisted procedures (sleeve gastrectomy, hysterectomy, cholecystectomy, and radical prostatectomy), we simulate real-world annotation scarcity by varying the proportion of available step annotations (<InlineEquation ID="IEq1"> <EquationSource Format="TEX">$\alpha $</EquationSource> <EquationSource Format="MATHML"><math> <mi>α</mi> </math></EquationSource> </InlineEquation> <InlineEquation ID="IEq2"> <EquationSource Format="TEX">$\in $</EquationSource> <EquationSource Format="MATHML"><math> <mo>∈</mo> </math></EquationSource> </InlineEquation> 0.25, 0.5, 0.75, 1.0). We benchmark the performance of a 2D CNN model trained with and without weak label pre-training. Results: Pre-training with surgical phase labels—particularly from the same procedure type (Phase-Within)—consistently improved step recognition performance, with gains up to 6.4 f1-score points over standard ImageNet-based models under limited annotation conditions (<InlineEquation ID="IEq3"> <EquationSource Format="TEX">$\alpha $</EquationSource> <EquationSource Format="MATHML"><math> <mi>α</mi> </math></EquationSource> </InlineEquation> = 0.25 on SLG). Cross-procedure step pre-training was beneficial for some procedures, and time-based labels provided moderate gains depending on procedure structure. Label efficiency analysis shows the baseline model would require labeling an additional 30–60 videos at <InlineEquation ID="IEq4"> <EquationSource Format="TEX">$\alpha $</EquationSource> <EquationSource Format="MATHML"><math> <mi>α</mi> </math></EquationSource> </InlineEquation> = 0.25 to match the performance achieved by the best weak-pretraining strategy across procedures. Conclusion: Weakly supervised pre-training offers a practical strategy to improve surgical step recognition when annotated data is scarce. This approach can support scalable feedback and assessment in surgical training workflows where comprehensive annotations are infeasible.

Weakly supervised pre-training for surgical step recognition using unannotated and heterogeneously labeled videos

期刊论文
出版模式：

开放获取
发表日期：

2025年12月2日

查看全文

Sreeram Kamabattula,
Kai Chen,
Kiran Bhattacharyya

摘要

Purpose:

Surgical video review is essential for minimally invasive surgical training, but manual annotation of surgical steps is time-consuming and limits scalability. We propose a weakly supervised pre-training framework that leverages unannotated or heterogeneously labeled surgical videos to improve automated surgical step recognition.

Methods:

We evaluate three types of weak labels derived from unannotated datasets: (1) surgical phases from the same or other procedures, (2) surgical steps from different procedure types, and (3) intraoperative time progression. Using datasets from four robotic-assisted procedures (sleeve gastrectomy, hysterectomy, cholecystectomy, and radical prostatectomy), we simulate real-world annotation scarcity by varying the proportion of available step annotations ( $\alpha $ $α$ $\in $ $\in$ 0.25, 0.5, 0.75, 1.0). We benchmark the performance of a 2D CNN model trained with and without weak label pre-training.

Results:

Pre-training with surgical phase labels—particularly from the same procedure type (Phase-Within)—consistently improved step recognition performance, with gains up to 6.4 f1-score points over standard ImageNet-based models under limited annotation conditions ( $\alpha $ $α$ = 0.25 on SLG). Cross-procedure step pre-training was beneficial for some procedures, and time-based labels provided moderate gains depending on procedure structure. Label efficiency analysis shows the baseline model would require labeling an additional 30–60 videos at $\alpha $ $α$ = 0.25 to match the performance achieved by the best weak-pretraining strategy across procedures.

Conclusion:

Weakly supervised pre-training offers a practical strategy to improve surgical step recognition when annotated data is scarce. This approach can support scalable feedback and assessment in surgical training workflows where comprehensive annotations are infeasible.