<p>Multi-modal fusion has emerged as a promising paradigm for modern object detection systems. However, existing works often involve substantial redundant computation during the stages of feature extraction, processing, and fusion, thereby limiting their ability to achieve both high accuracy and real-time performance. To this end, we propose <Emphasis FontCategory="NonProportional">Eagle</Emphasis>, a real-time, high-accuracy detection system that integrates camera and millimeter-wave radar data via the three-level entropy-driven mechanism, which consists of three key components: (i) an <i>entropy measurement-based feature extraction</i>, which quantifies feature uncertainty through entropy computation, generating a weight matrix to adjust feature channels, followed by enhancing the discrimination of informative features while suppressing redundant ones; (ii) an <i>entropy-contrast-based proposal selection</i>, which utilizes the similarity between anchor boxes from camera and radar modalities to calculate a contrast entropy value, where low-quality proposals are filtered based on this metric, thereby reducing both false positives and false negatives; (iii) an <i>entropy-guided-based feature fusion</i>, which dynamically weights the feature mapping according to entropy values, achieving effective multi-modal integration through spatial-level interactions. Experimental results demonstrate that <Emphasis FontCategory="NonProportional">Eagle</Emphasis> improves the mean average precision (mAP) and nuScenes detection score (NDS) by 2.3% and 4.4%, respectively, compared to state-of-the-art baselines.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Eagle: three-level entropy-driven object detection via mmWave radar and camera fusion

  • Ling Xing,
  • Rongrong Wang,
  • Kaikai Deng,
  • Jianping Gao,
  • Honghai Wu,
  • Huahong Ma

摘要

Multi-modal fusion has emerged as a promising paradigm for modern object detection systems. However, existing works often involve substantial redundant computation during the stages of feature extraction, processing, and fusion, thereby limiting their ability to achieve both high accuracy and real-time performance. To this end, we propose Eagle, a real-time, high-accuracy detection system that integrates camera and millimeter-wave radar data via the three-level entropy-driven mechanism, which consists of three key components: (i) an entropy measurement-based feature extraction, which quantifies feature uncertainty through entropy computation, generating a weight matrix to adjust feature channels, followed by enhancing the discrimination of informative features while suppressing redundant ones; (ii) an entropy-contrast-based proposal selection, which utilizes the similarity between anchor boxes from camera and radar modalities to calculate a contrast entropy value, where low-quality proposals are filtered based on this metric, thereby reducing both false positives and false negatives; (iii) an entropy-guided-based feature fusion, which dynamically weights the feature mapping according to entropy values, achieving effective multi-modal integration through spatial-level interactions. Experimental results demonstrate that Eagle improves the mean average precision (mAP) and nuScenes detection score (NDS) by 2.3% and 4.4%, respectively, compared to state-of-the-art baselines.