<p>In recent years, LiDAR-Camera fusion for 3D object detection has emerged as a research hotspot, owing to its superior performance compared to single-sensor approaches. However, the vast majority of methods adopt fusion in the point, voxel, or BEV domains, which may lead to domain discrepancies and misalignment, resulting in suboptimal detection accuracy. In contrast, this paper introduces a novel range-view fusion strategy, leveraging its unique advantages—data homogeneity, accurate feature alignment, and high efficiency—over other methods. We introduce the Range-view Pyramidal Fusion network (RPfusion) as a plug-and-play framework compatible with various LiDAR-based detectors. It incorporates two dedicated modules—Appearance Feature Fusion (AFF) and Range-Guided Cross-Layer Modulation (RGCLM)—to effectively tackle the inherent challenges of range-view fusion: substantial object scale variations and feature vanishment caused by occlusion. First, the AFF module adaptively integrates complementary geometric and information across different scales using dilated cross-attention. Next, RGCLM module is proposed to extract rich contextual information for comprehensive perception and enhance features of occluded targets. The output features can globally decorate raw point clouds to boost detection accuracy of any LiDAR-based model. Built upon RPFusion, we further develop a novel two-stage detector—RPfusion-Det that additionally incorporates an RoIfusion module for fine-grained local feature aggregation. Extensive experiments on the KITTI and NuScenes datasets demonstrate that RPFusion can consistently enhances various detection models, while RPfusion-Det achieves superior accuracy with efficient inference.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Lidar-camera range-view fusion for 3D object detection in autonomous driving

  • Xuzhong Hu,
  • Zaipeng Duan,
  • Pei An,
  • Jun Zhang,
  • Jie Ma

摘要

In recent years, LiDAR-Camera fusion for 3D object detection has emerged as a research hotspot, owing to its superior performance compared to single-sensor approaches. However, the vast majority of methods adopt fusion in the point, voxel, or BEV domains, which may lead to domain discrepancies and misalignment, resulting in suboptimal detection accuracy. In contrast, this paper introduces a novel range-view fusion strategy, leveraging its unique advantages—data homogeneity, accurate feature alignment, and high efficiency—over other methods. We introduce the Range-view Pyramidal Fusion network (RPfusion) as a plug-and-play framework compatible with various LiDAR-based detectors. It incorporates two dedicated modules—Appearance Feature Fusion (AFF) and Range-Guided Cross-Layer Modulation (RGCLM)—to effectively tackle the inherent challenges of range-view fusion: substantial object scale variations and feature vanishment caused by occlusion. First, the AFF module adaptively integrates complementary geometric and information across different scales using dilated cross-attention. Next, RGCLM module is proposed to extract rich contextual information for comprehensive perception and enhance features of occluded targets. The output features can globally decorate raw point clouds to boost detection accuracy of any LiDAR-based model. Built upon RPFusion, we further develop a novel two-stage detector—RPfusion-Det that additionally incorporates an RoIfusion module for fine-grained local feature aggregation. Extensive experiments on the KITTI and NuScenes datasets demonstrate that RPFusion can consistently enhances various detection models, while RPfusion-Det achieves superior accuracy with efficient inference.