MT-Occ: Single-View 3D Occupancy Prediction via Multi-task Distillation
摘要
3D occupancy prediction is gaining traction in autonomous driving for its ability to jointly model environment geometry and semantics. Weakly supervised methods learn 3D representations solely from multi-view 2D labels, making them ideal for data-scarce scenarios. However, distilling 2D knowledge into 3D is challenging due to limited information and noisy pseudo labels. To tackle the above challenges, we introduce MT-Occ, a single-view self-supervised 3D occupancy prediction method that enhances 2D-to-3D distillation by leveraging pretrained multi-task features and modelling task interactions. Our approach includes three effective and flexible components: 1) an effective fusion technique utilising pretrained features of multiple relevant tasks, 2) a spatial cross-task attention module for geometric-semantic distillation and 3) a view-consistent label refinement strategy to improve 2D pseudo labels. MT-Occ achieves state-of-the-art results on autonomous driving benchmarks, outperforming prior work with +16.04% relative mIoU on SSCBench-KITTI-360 and +33.33% on SSCBench-nuScenes, particularly excelling on safety-critical small classes. Extensive experiments validate the effectiveness and flexibility of our design choices.