This paper presents Learn Where I Can Walk (LWICW), a novel auto-labeling approach for segmenting walked areas using a trajectory estimated from multiple sequential monocular camera images, aimed at training supervised segmentation models for the navigation of visually impaired people. The proposed method uses images sourced from Mapillary, which is a collaborative platform to share street-level images. The approach involves extracting the walked path of the camera operator through the camera poses, the filtering of occluded walking path poses using Depth Anything V2, and the application of Segment Anything Model 2 (SAM 2) for segmentation. The LWICW auto-labels are validated against a manually labeled dataset from Mapillary and compared to the state-of-the-art zero-shot segmentation model Grounded SAM 2. The LWICW method achieves an overall mean Intersection over Union (mIoU) of 93.9% and a mean \(\text {F}_1\) score ( \(\text {mF}_1\) ) of 96.6%, which represents a performance improvement of +1.6% points on mIoU and +1.0% points on \(\text {mF}_1\) compared to the Grounded SAM 2 approach.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Learn Where I Can Walk: Auto-labeling of Walked Areas Using Monocular Camera Trajectory

  • Helmut Engelhardt,
  • Matthias Kalenberg,
  • Jörg Franke,
  • Sina Martin

摘要

This paper presents Learn Where I Can Walk (LWICW), a novel auto-labeling approach for segmenting walked areas using a trajectory estimated from multiple sequential monocular camera images, aimed at training supervised segmentation models for the navigation of visually impaired people. The proposed method uses images sourced from Mapillary, which is a collaborative platform to share street-level images. The approach involves extracting the walked path of the camera operator through the camera poses, the filtering of occluded walking path poses using Depth Anything V2, and the application of Segment Anything Model 2 (SAM 2) for segmentation. The LWICW auto-labels are validated against a manually labeled dataset from Mapillary and compared to the state-of-the-art zero-shot segmentation model Grounded SAM 2. The LWICW method achieves an overall mean Intersection over Union (mIoU) of 93.9% and a mean \(\text {F}_1\) score ( \(\text {mF}_1\) ) of 96.6%, which represents a performance improvement of +1.6% points on mIoU and +1.0% points on \(\text {mF}_1\) compared to the Grounded SAM 2 approach.