In autonomous driving, vision-based 3D occupancy perception has received widespread attention for accurately representing 3D scenes compared to Bird’s-Eye-View (BEV) representations. A key step in vision-based 3D occupancy prediction involves view transformation, which converts 2D image features into a 3D voxel representation. Currently, the two most prominent view transformation paradigms are forward projection based on depth estimation and backward projection based on transformer. However, these two paradigms face inherent limitations. The former produces sparse voxel features, while the latter often focuses on unoccupied regions because of the absence of geometric prior information. To address these challenges, we propose JP-Occ, a joint forward-backward projection occupancy prediction framework that leverages the strengths of both paradigms to obtain high-quality voxel features. Our framework first employs an Occupancy-Aware Forward Projection module to generate initial voxel features and query proposals, followed by a Geometry-Guided Backward Projection module to enrich and densify the voxel features. In addition, we introduce a plug-and-play BEV Feature Aggregation module to enhance the global representation of voxel features. Experiments conducted on the Occ3D-nuScenes dataset demonstrate that JP-Occ achieves promising performance with high computational efficiency, underscoring the effectiveness and superiority of our approach.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

JP-Occ: 3D Occupancy Prediction with View Transformation Based on Joint Projection

  • Wei Liu,
  • Guixing Xu,
  • Zuotao Ning,
  • Qi Guo,
  • Junxin Jin,
  • Biao Xu,
  • Shuai Cheng,
  • Liuyu Pei

摘要

In autonomous driving, vision-based 3D occupancy perception has received widespread attention for accurately representing 3D scenes compared to Bird’s-Eye-View (BEV) representations. A key step in vision-based 3D occupancy prediction involves view transformation, which converts 2D image features into a 3D voxel representation. Currently, the two most prominent view transformation paradigms are forward projection based on depth estimation and backward projection based on transformer. However, these two paradigms face inherent limitations. The former produces sparse voxel features, while the latter often focuses on unoccupied regions because of the absence of geometric prior information. To address these challenges, we propose JP-Occ, a joint forward-backward projection occupancy prediction framework that leverages the strengths of both paradigms to obtain high-quality voxel features. Our framework first employs an Occupancy-Aware Forward Projection module to generate initial voxel features and query proposals, followed by a Geometry-Guided Backward Projection module to enrich and densify the voxel features. In addition, we introduce a plug-and-play BEV Feature Aggregation module to enhance the global representation of voxel features. Experiments conducted on the Occ3D-nuScenes dataset demonstrate that JP-Occ achieves promising performance with high computational efficiency, underscoring the effectiveness and superiority of our approach.