Scholarlink

The Segment Anything Model (SAM) is a powerful foundational model designed to generalize and automate image segmentation tasks. However, when directly applied to surgical images, SAM encounters several critical challenges including the imaging disparities between natural and medical images, its reliance on high-quality prompts, and the lack of support for multi-class segmentation. To address these limitations, we propose a novel approach Detection-SAM (DetSAM), which integrates a detection module with SAM. By leveraging the output of a detection module as a prompt for each individual class, fully automatic multi-class surgical instrument segmentation can be realized. Meanwhile, we also design a block-wise fine-tuned image encoder to extract image features. We have validated DetSAM on two standard public benchmark datasets (i.e., Endovis2017 and Endovis2018), achieving Ch_IoU scores of 77.53% and 76.61% respectively, surpassing existing state-of-the-art (SOTA) methods. The superior performances underscore the effectiveness of combining our detection module and SAM for surgical instrument segmentation. The code is available at: https://github.com/JingsongWang04/DetSAM

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

DetSAM: A Joint Detection-and-Segmentation Learning Framework for Multi-class Surgical Instrument Segmentation

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Jingsong Wang,
  • Rui Song,
  • Yibin Li,
  • Max Q.-H. Meng,
  • Zhe Min

摘要

The Segment Anything Model (SAM) is a powerful foundational model designed to generalize and automate image segmentation tasks. However, when directly applied to surgical images, SAM encounters several critical challenges including the imaging disparities between natural and medical images, its reliance on high-quality prompts, and the lack of support for multi-class segmentation. To address these limitations, we propose a novel approach Detection-SAM (DetSAM), which integrates a detection module with SAM. By leveraging the output of a detection module as a prompt for each individual class, fully automatic multi-class surgical instrument segmentation can be realized. Meanwhile, we also design a block-wise fine-tuned image encoder to extract image features. We have validated DetSAM on two standard public benchmark datasets (i.e., Endovis2017 and Endovis2018), achieving Ch_IoU scores of 77.53% and 76.61% respectively, surpassing existing state-of-the-art (SOTA) methods. The superior performances underscore the effectiveness of combining our detection module and SAM for surgical instrument segmentation. The code is available at: https://github.com/JingsongWang04/DetSAM