<p>Multimodal 3D object detection, fusing LiDAR and camera data, is crucial for autonomous driving. However, existing methods suffer significant performance degradation in adverse weather due to feature mismatches caused by compromised sensor data. For instance, LiDAR is obscured by snow, while cameras struggle in low light. State-of-the-art approaches, while accurate in ideal conditions, lack robustness for real-world applications. To address this, we introduce a comprehensive benchmark to evaluate model robustness across four adverse weather conditions. We then propose a synergistic framework combining robust fusion design and data augmentation to mitigate asymmetric sensor degradation. Extensive experiments show our framework effectively handles imbalanced feature degradation, significantly improving system robustness. Our findings provide critical insights for optimizing multimodal perception in autonomous driving under adverse weather conditions, paving a more reliable pathway for real-world deployment.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Robust multimodal 3D object detection: overcoming weather challenges in autonomous driving perception systems

  • Runhong Dong,
  • Ran Zhou,
  • Fangchao Xu,
  • Junjie Jin,
  • Feng Sun,
  • Shengyuan Jiang,
  • Zhanwei Bai,
  • Xiaoyou Zhang

摘要

Multimodal 3D object detection, fusing LiDAR and camera data, is crucial for autonomous driving. However, existing methods suffer significant performance degradation in adverse weather due to feature mismatches caused by compromised sensor data. For instance, LiDAR is obscured by snow, while cameras struggle in low light. State-of-the-art approaches, while accurate in ideal conditions, lack robustness for real-world applications. To address this, we introduce a comprehensive benchmark to evaluate model robustness across four adverse weather conditions. We then propose a synergistic framework combining robust fusion design and data augmentation to mitigate asymmetric sensor degradation. Extensive experiments show our framework effectively handles imbalanced feature degradation, significantly improving system robustness. Our findings provide critical insights for optimizing multimodal perception in autonomous driving under adverse weather conditions, paving a more reliable pathway for real-world deployment.