Surgical scene understanding in Minimally Invasive Surgery (MIS) is crucial for advancing Computer-Assisted Intervention (CAI) applications, enhancing surgical safety, and improving navigation. This work introduces a novel multi-task learning framework that jointly performs binary surgical tool segmentation and monocular depth estimation in laparoscopic surgical scenes. The framework employs a staged learning strategy: first, leveraging widely available tool segmentation datasets to pre-train the network, followed by multi-task training using pseudo-masks and self-supervised monocular depth estimation. Extensive experiments demonstrate the effectiveness of the proposed framework, achieving competitive performance on depth estimation compared to state-of-the-art methods. Validation on two publicly available datasets highlights its robustness and adaptability across diverse surgical scenarios. These results emphasize the potential of multi-task learning to advance laparoscopic surgical perception. The implementation is available on  GitHub .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Surgical Instrument Segmentation and Self-Supervised Monocular Depth Estimation in Minimally Invasive Surgery: A Multi-task Learning Approach

  • Stefano Mazzocchetti,
  • Laura Cercenelli,
  • Emanuela Marcelli

摘要

Surgical scene understanding in Minimally Invasive Surgery (MIS) is crucial for advancing Computer-Assisted Intervention (CAI) applications, enhancing surgical safety, and improving navigation. This work introduces a novel multi-task learning framework that jointly performs binary surgical tool segmentation and monocular depth estimation in laparoscopic surgical scenes. The framework employs a staged learning strategy: first, leveraging widely available tool segmentation datasets to pre-train the network, followed by multi-task training using pseudo-masks and self-supervised monocular depth estimation. Extensive experiments demonstrate the effectiveness of the proposed framework, achieving competitive performance on depth estimation compared to state-of-the-art methods. Validation on two publicly available datasets highlights its robustness and adaptability across diverse surgical scenarios. These results emphasize the potential of multi-task learning to advance laparoscopic surgical perception. The implementation is available on  GitHub .