<p>An ultralightweight self-supervised stereo matching network, called US<InlineEquation ID="IEq3"> <EquationSource Format="TEX">\(^3\)</EquationSource> <EquationSource Format="MATHML"><math> <mmultiscripts> <mrow /> <mrow /> <mn>3</mn> </mmultiscripts> </math></EquationSource> </InlineEquation>Net, which requires only 12K parameters, is designed for efficient and accurate depth estimation using resource-constrained devices. US<InlineEquation ID="IEq4"> <EquationSource Format="TEX">\(^3\)</EquationSource> <EquationSource Format="MATHML"><math> <mmultiscripts> <mrow /> <mrow /> <mn>3</mn> </mmultiscripts> </math></EquationSource> </InlineEquation>Net incorporates two key innovations: a low-complexity feature extraction module and a soft occlusion detection approach for performance improvement. First, we design a low-complexity feature extraction module to reduce the computational burden while preserving structural details necessary for stereo matching. By refining the encoder backbone and aggregation module, our design ensures a better balance between model complexity and accuracy. Second, to address occlusion-related errors in disparity estimation, we propose a novel occlusion detection method, called Depth-Aware Geometric Soft Occlusion (DAGSO), to adaptively define the occlusion confidence scores based on depth information. DAGSO can effectively mitigate false occlusions in distant regions and can improve the accuracy of disparity estimation. Experimental results using KITTI datasets demonstrate that US<InlineEquation ID="IEq5"> <EquationSource Format="TEX">\(^3\)</EquationSource> <EquationSource Format="MATHML"><math> <mmultiscripts> <mrow /> <mrow /> <mn>3</mn> </mmultiscripts> </math></EquationSource> </InlineEquation>Net achieves state-of-the-art performance in terms of model complexity and depth estimation accuracy. It outperforms previous self-supervised stereo matching methods and monocular depth estimation methods in metrics such as AbsRel, SqRel, RMSE, and RMSElog at a reduction of parameter size by 47% compared with ES<InlineEquation ID="IEq6"> <EquationSource Format="TEX">\(^3\)</EquationSource> <EquationSource Format="MATHML"><math> <mmultiscripts> <mrow /> <mrow /> <mn>3</mn> </mmultiscripts> </math></EquationSource> </InlineEquation>Net (23K). This makes US<InlineEquation ID="IEq7"> <EquationSource Format="TEX">\(^3\)</EquationSource> <EquationSource Format="MATHML"><math> <mmultiscripts> <mrow /> <mrow /> <mn>3</mn> </mmultiscripts> </math></EquationSource> </InlineEquation>Net a practical solution for real-time depth estimation on edge devices such as drones and autonomous systems. Code is available at: <a href="https://github.com/g830319ag/US3Net.">https://github.com/g830319ag/US3Net.</a></p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

US\(^3\)Net: Ultralightweight Self-Supervised Stereo Matching Network using Depth-Aware Geometric Soft Occlusion

  • Po-Chung Jen,
  • Tzu-Chi Liu,
  • I-Sheng Fang,
  • Hsiao-Chieh Wen,
  • Chia-Lun Hsu,
  • Ping-Yang Chen,
  • Chang-Hsing Lee,
  • Yong-Sheng Chen

摘要

An ultralightweight self-supervised stereo matching network, called US \(^3\) 3 Net, which requires only 12K parameters, is designed for efficient and accurate depth estimation using resource-constrained devices. US \(^3\) 3 Net incorporates two key innovations: a low-complexity feature extraction module and a soft occlusion detection approach for performance improvement. First, we design a low-complexity feature extraction module to reduce the computational burden while preserving structural details necessary for stereo matching. By refining the encoder backbone and aggregation module, our design ensures a better balance between model complexity and accuracy. Second, to address occlusion-related errors in disparity estimation, we propose a novel occlusion detection method, called Depth-Aware Geometric Soft Occlusion (DAGSO), to adaptively define the occlusion confidence scores based on depth information. DAGSO can effectively mitigate false occlusions in distant regions and can improve the accuracy of disparity estimation. Experimental results using KITTI datasets demonstrate that US \(^3\) 3 Net achieves state-of-the-art performance in terms of model complexity and depth estimation accuracy. It outperforms previous self-supervised stereo matching methods and monocular depth estimation methods in metrics such as AbsRel, SqRel, RMSE, and RMSElog at a reduction of parameter size by 47% compared with ES \(^3\) 3 Net (23K). This makes US \(^3\) 3 Net a practical solution for real-time depth estimation on edge devices such as drones and autonomous systems. Code is available at: https://github.com/g830319ag/US3Net.