Autonomous vehicles depend on complementary sensors—cameras, LiDAR, and radar—to perceive their environment, yet adverse weather such as heavy rain, fog, or low-light can severely degrade individual modalities. We introduce PREFUSE, a Probabilistic Reliability-Enhanced Fusion framework that adapts to real-time environmental and sensor quality cues to maintain robust 3D object detection. PREFUSE employs an ensemble-based weather classifier—trained on a diverse synthetic-weather dataset collected in CARLA and subsequently fine-tuned on a limited set of real-world weather samples from nuScenes—to infer current conditions. The classifier’s outputs feed into a latent reliability model that captures both weather-induced and sensor-specific degradation patterns. Sensor features are first extracted by dedicated convolutional networks, then dynamically re-weighted via an attention-based gating mechanism informed by the reliability model. A Bayesian fusion module integrates these weighted features at the mid-level, explicitly quantifying uncertainty. In extensive experiments on the nuScenes dataset, PREFUSE achieves mAP@50 of about 0.70 across diverse weather conditions, sustaining under 10% performance degradation in adverse scenarios—outperforming static fusion baselines by over 5%—while running at real-time speeds of 18 Hz on automotive-grade hardware. Although PREFUSE is generalizable to any downstream computer-vision task, we demonstrate its effectiveness specifically on 3D object detection.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

PREFUSE: A Probabilistic Reliability-Weighted Fusion for Weather-Aware Perception in Autonomous Vehicles

  • Andrews Tang,
  • Issa W. AlHmoud,
  • Balakrishna Gokaraju,
  • Chyi Lyi Liang

摘要

Autonomous vehicles depend on complementary sensors—cameras, LiDAR, and radar—to perceive their environment, yet adverse weather such as heavy rain, fog, or low-light can severely degrade individual modalities. We introduce PREFUSE, a Probabilistic Reliability-Enhanced Fusion framework that adapts to real-time environmental and sensor quality cues to maintain robust 3D object detection. PREFUSE employs an ensemble-based weather classifier—trained on a diverse synthetic-weather dataset collected in CARLA and subsequently fine-tuned on a limited set of real-world weather samples from nuScenes—to infer current conditions. The classifier’s outputs feed into a latent reliability model that captures both weather-induced and sensor-specific degradation patterns. Sensor features are first extracted by dedicated convolutional networks, then dynamically re-weighted via an attention-based gating mechanism informed by the reliability model. A Bayesian fusion module integrates these weighted features at the mid-level, explicitly quantifying uncertainty. In extensive experiments on the nuScenes dataset, PREFUSE achieves mAP@50 of about 0.70 across diverse weather conditions, sustaining under 10% performance degradation in adverse scenarios—outperforming static fusion baselines by over 5%—while running at real-time speeds of 18 Hz on automotive-grade hardware. Although PREFUSE is generalizable to any downstream computer-vision task, we demonstrate its effectiveness specifically on 3D object detection.