<p>Depth completion aims to recover a dense depth map from a sparse depth map and its corresponding image. This task is inherently challenged by two critical issues: (1) how to effectively fuse image and depth information and (2) how to enhance depth details for higher accuracy. We propose BCAP-Net, a Bidirectional Cross-fusion and Adaptive Propagation Network, to overcome these limitations. It incorporates three innovative components: First, we propose a recurrent cross fusion (RCF) encoder with a bidirectional residual cross-fusion module (BRCFM) that facilitates deep feature fusion through innovative cross-position and cross-attention mechanisms, ensuring effective interaction between depth and image modalities. Second, to better recover the depth map, we propose a hierarchical multi-scale depth-aware encoder (HMDA), which downsamples the sparse depth map to the dense depth map at different scales through weighted pooling, and completes the reconstruction by combining the depth-encoded features. Third, to enhance depth details, the depth confidence propagation refiner (DCPR) is proposed to adaptively adjust the depth update weights by leveraging the depth hidden features and depth differences, thereby refining local details and improving overall depth accuracy. Experimental results on the KITTI, NYU Depth V2, and VOID datasets demonstrate the superiority of the proposed method compared with state-of-the-art approaches.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Bcap-net: a bidirectional cross-fusion and adaptive depth propagation network for depth completion

  • Han Chen,
  • Yongxiong Wang,
  • Shuwen Jia,
  • Xiao Jin,
  • Jiayi Zhang

摘要

Depth completion aims to recover a dense depth map from a sparse depth map and its corresponding image. This task is inherently challenged by two critical issues: (1) how to effectively fuse image and depth information and (2) how to enhance depth details for higher accuracy. We propose BCAP-Net, a Bidirectional Cross-fusion and Adaptive Propagation Network, to overcome these limitations. It incorporates three innovative components: First, we propose a recurrent cross fusion (RCF) encoder with a bidirectional residual cross-fusion module (BRCFM) that facilitates deep feature fusion through innovative cross-position and cross-attention mechanisms, ensuring effective interaction between depth and image modalities. Second, to better recover the depth map, we propose a hierarchical multi-scale depth-aware encoder (HMDA), which downsamples the sparse depth map to the dense depth map at different scales through weighted pooling, and completes the reconstruction by combining the depth-encoded features. Third, to enhance depth details, the depth confidence propagation refiner (DCPR) is proposed to adaptively adjust the depth update weights by leveraging the depth hidden features and depth differences, thereby refining local details and improving overall depth accuracy. Experimental results on the KITTI, NYU Depth V2, and VOID datasets demonstrate the superiority of the proposed method compared with state-of-the-art approaches.