Learn Where I Can Walk: Auto-labeling of Walked Areas Using Monocular Camera Trajectory
摘要
This paper presents Learn Where I Can Walk (LWICW), a novel auto-labeling approach for segmenting walked areas using a trajectory estimated from multiple sequential monocular camera images, aimed at training supervised segmentation models for the navigation of visually impaired people. The proposed method uses images sourced from Mapillary, which is a collaborative platform to share street-level images. The approach involves extracting the walked path of the camera operator through the camera poses, the filtering of occluded walking path poses using Depth Anything V2, and the application of Segment Anything Model 2 (SAM 2) for segmentation. The LWICW auto-labels are validated against a manually labeled dataset from Mapillary and compared to the state-of-the-art zero-shot segmentation model Grounded SAM 2. The LWICW method achieves an overall mean Intersection over Union (mIoU) of 93.9% and a mean \(\text {F}_1\) score ( \(\text {mF}_1\) ) of 96.6%, which represents a performance improvement of +1.6% points on mIoU and +1.0% points on \(\text {mF}_1\) compared to the Grounded SAM 2 approach.