Semi-supervised Instance Segmentation (SSIS) amalgamates labeled and unlabeled data to bolster training efficiency. However, conventional SSIS methodologies, which depend exclusively on RGB imagery for pseudo-labeling, are susceptible to inaccuracies under challenging conditions such as low-light environments and occlusions. In this study, we demonstrate that depth maps offer a more robust representation of individual instances by capitalizing on their distance values, thereby enabling more accurate contour delineation under these imaging conditions. Despite the advantages, incorporating depth information into SSIS presents a challenge due to the inherent differences between the RGB and depth image domains. To address this, we introduce Depth-Guided Semi-Supervised Instance Segmentation (DGIS), an innovative framework that incorporates depth maps into the training regimen. Specifically, we propose a Depth Feature Fusion (DFF) module that seamlessly integrates features derived from both depth estimation maps and RGB images. Additionally, we develop a Depth Controller (DC) module designed to dynamically modulate the influence of depth information, enhancing model adaptability and hastening convergence. Our extensive experiments on the COCO and Cityscapes datasets underscore the superiority of DGIS over current state-of-the-art methods. With only 1%, 5%, and 10% labeled data, DGIS achieves mean Average Precision (mAP) scores of 22.29%, 31.47%, and 35.14%, respectively. Code is available at: https://github.com/CStar-777/Depth-Guided .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhancing Semi-Supervised Instance Segmentation via Depth-Guided Learning

  • Xin Chen,
  • Jie Hu,
  • Yan Zhang

摘要

Semi-supervised Instance Segmentation (SSIS) amalgamates labeled and unlabeled data to bolster training efficiency. However, conventional SSIS methodologies, which depend exclusively on RGB imagery for pseudo-labeling, are susceptible to inaccuracies under challenging conditions such as low-light environments and occlusions. In this study, we demonstrate that depth maps offer a more robust representation of individual instances by capitalizing on their distance values, thereby enabling more accurate contour delineation under these imaging conditions. Despite the advantages, incorporating depth information into SSIS presents a challenge due to the inherent differences between the RGB and depth image domains. To address this, we introduce Depth-Guided Semi-Supervised Instance Segmentation (DGIS), an innovative framework that incorporates depth maps into the training regimen. Specifically, we propose a Depth Feature Fusion (DFF) module that seamlessly integrates features derived from both depth estimation maps and RGB images. Additionally, we develop a Depth Controller (DC) module designed to dynamically modulate the influence of depth information, enhancing model adaptability and hastening convergence. Our extensive experiments on the COCO and Cityscapes datasets underscore the superiority of DGIS over current state-of-the-art methods. With only 1%, 5%, and 10% labeled data, DGIS achieves mean Average Precision (mAP) scores of 22.29%, 31.47%, and 35.14%, respectively. Code is available at: https://github.com/CStar-777/Depth-Guided .