Nuclei instance segmentation is crucial for biomedical research and disease diagnosis. Pathologists utilize information such as color, shape, and the surrounding tissue microenvironment to distinguish nuclei. However, existing models are limited as they rely solely on features from the current patch, neglecting contextual information from neighboring patches. This limitation impedes the model’s ability to accurately identify nuclei. To address this issue, we propose CA-SAM2, a novel framework that enhances the prompt propagation capability of the Segment Anything Model 2 (SAM2) through a Context Injection Module(CIM), integrating surrounding contextual information during segmentation. Additionally, to adapt SAM2 to the pathology image domain, we introduce a convolutional branch to extract domain-specific features from pathological images. We further design a Multi-Level Feature Refinement Block (MFRB) to refine the prior features extracted by SAM2 and integrate domain features. Finally, we incorporate a regression head and a classification head after the convolutional branch to automatically generate point prompts, eliminating the need for manual annotation. Extensive evaluations of CA-SAM2 on the MoNuSeg and CPM-17 datasets demonstrate its effectiveness and practicality in enhancing nuclei segmentation. The code is available at https://github.com/HanbinHuang123/CA-SAM2 .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

CA-SAM2: SAM2-Based Context-Aware Network with Auto-prompting for Nuclei Instance Segmentation

  • Hanbin Huang,
  • Hongliang He,
  • Liying Xu,
  • Xudong Zhu,
  • Siwei Feng,
  • Guohong Fu

摘要

Nuclei instance segmentation is crucial for biomedical research and disease diagnosis. Pathologists utilize information such as color, shape, and the surrounding tissue microenvironment to distinguish nuclei. However, existing models are limited as they rely solely on features from the current patch, neglecting contextual information from neighboring patches. This limitation impedes the model’s ability to accurately identify nuclei. To address this issue, we propose CA-SAM2, a novel framework that enhances the prompt propagation capability of the Segment Anything Model 2 (SAM2) through a Context Injection Module(CIM), integrating surrounding contextual information during segmentation. Additionally, to adapt SAM2 to the pathology image domain, we introduce a convolutional branch to extract domain-specific features from pathological images. We further design a Multi-Level Feature Refinement Block (MFRB) to refine the prior features extracted by SAM2 and integrate domain features. Finally, we incorporate a regression head and a classification head after the convolutional branch to automatically generate point prompts, eliminating the need for manual annotation. Extensive evaluations of CA-SAM2 on the MoNuSeg and CPM-17 datasets demonstrate its effectiveness and practicality in enhancing nuclei segmentation. The code is available at https://github.com/HanbinHuang123/CA-SAM2 .