Precise localization of surgical instrument tips is essential for evaluating fine motor skills and enabling automation in microsurgical training. This study presents a deep learning framework based on keypoint heatmap regression to detect instrument tips in frames extracted from simulated surgical videos. A dataset of 1781 annotated frames from seven videos was used for evaluation. The framework was trained with different loss functions—root mean squared error (RMSE), weighted Kullback-Leibler divergence (WKLD), and Dice loss—and compared with direct coordinate regression and segmentation-based models. The RMSE-based model achieved the best performance (MAE = 7.54 pixels), while the WKLD-based model provided more stable predictions across thresholds for blank mask detection. Segmentation and direct regression models showed significantly higher errors. Statistical analyses confirmed the advantage of heatmap regression over baseline approaches. These results support the adoption of heatmap-based keypoint localization for robust tool tracking in simulated surgical environments and its integration into training systems for skill assessment.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

AI-Driven Surgical Tool Localization in Microsurgical Training Simulations

  • Flavio Di Lisio,
  • Angelo Lasala,
  • Francesca Pia Villani,
  • Olimpia Mani,
  • Andrea Poggetti,
  • Sandra Pfanner,
  • Marina Carbone,
  • Paolo Domenico Parchi,
  • Emanuele Frontoni,
  • Sara Moccia

摘要

Precise localization of surgical instrument tips is essential for evaluating fine motor skills and enabling automation in microsurgical training. This study presents a deep learning framework based on keypoint heatmap regression to detect instrument tips in frames extracted from simulated surgical videos. A dataset of 1781 annotated frames from seven videos was used for evaluation. The framework was trained with different loss functions—root mean squared error (RMSE), weighted Kullback-Leibler divergence (WKLD), and Dice loss—and compared with direct coordinate regression and segmentation-based models. The RMSE-based model achieved the best performance (MAE = 7.54 pixels), while the WKLD-based model provided more stable predictions across thresholds for blank mask detection. Segmentation and direct regression models showed significantly higher errors. Statistical analyses confirmed the advantage of heatmap regression over baseline approaches. These results support the adoption of heatmap-based keypoint localization for robust tool tracking in simulated surgical environments and its integration into training systems for skill assessment.