Multi-view human pose estimation aims to localize 3D skeleton joints in a synchronized and calibrated multi-view setup. Recent advances demonstrate that cross-view feature fusion significantly improves 3D coordinate localization accuracy. However, despite their promising results, conventional epipolar-based cross-view fusion methods suffer from two major limitations: (1) incomplete spatial sampling, which overlooks many informative features; and (2) high sensitivity to calibration errors, which compromises robustness in real-world scenarios with imperfect camera parameters. To address these challenges, we propose an epipolar-guided deformable feature fusion framework. Specifically, we first identify high-similarity keypoints along the epipolar line. These keypoints are then used as anchors for adaptive sampling of features from their surrounding regions, thereby alleviating the problem of incomplete spatial fusion. We further extend epipolar fusion into the temporal domain to enhance robustness against slight camera miscalibrations commonly found in real-world environments. In addition, we embed camera parameters into the feature representation to facilitate better perception of 3D cues. Extensive experiments on the Human3.6M and Ski-Pose datasets demonstrate the effectiveness of our approach. Under the condition of using two views, our method outperforms SOTA by 1.2mm and achieves MPJPE 24.6mm on the Human3.6M dataset.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Deformable Epipolar Transformer for Robust 3D Human Pose Estimation

  • Yang Gao,
  • Xiaoqi An,
  • Di Wang,
  • Fei Gao,
  • Lin Zhao

摘要

Multi-view human pose estimation aims to localize 3D skeleton joints in a synchronized and calibrated multi-view setup. Recent advances demonstrate that cross-view feature fusion significantly improves 3D coordinate localization accuracy. However, despite their promising results, conventional epipolar-based cross-view fusion methods suffer from two major limitations: (1) incomplete spatial sampling, which overlooks many informative features; and (2) high sensitivity to calibration errors, which compromises robustness in real-world scenarios with imperfect camera parameters. To address these challenges, we propose an epipolar-guided deformable feature fusion framework. Specifically, we first identify high-similarity keypoints along the epipolar line. These keypoints are then used as anchors for adaptive sampling of features from their surrounding regions, thereby alleviating the problem of incomplete spatial fusion. We further extend epipolar fusion into the temporal domain to enhance robustness against slight camera miscalibrations commonly found in real-world environments. In addition, we embed camera parameters into the feature representation to facilitate better perception of 3D cues. Extensive experiments on the Human3.6M and Ski-Pose datasets demonstrate the effectiveness of our approach. Under the condition of using two views, our method outperforms SOTA by 1.2mm and achieves MPJPE 24.6mm on the Human3.6M dataset.