<p>Transformer-based methods have recently made significant advances in 3D human pose estimation, demonstrating excellent performance in global feature extraction and long-range dependency modeling. However, these approaches still face limitations in refining local features and capturing local dependencies. To address this issue, we propose a novel convolution-enhanced dual-stream Transformer framework (DMFormer) designed to model human dependencies with multiple levels of granularity. This increases estimation accuracy through richer pose representations and more accurate motion correlation capture between keypoints. Specifically, DMFormer consists of three main components: (1) a Joint Transformer Encoder, which models spatiotemporal dependencies at the joint level; (2) a Part Transformer Encoder, which focuses on extracting spatiotemporal features at the part level, by leveraging convolutional operations to capture local details and model both global and local dependencies; and (3) a feature fusion layer which integrates multigranularity information to enrich and unify the feature representations. Extensive experiments conducted on two widely used 3D human pose estimation benchmarks, Human3.6M and MPI-INF-3DHP, demonstrate that DMFormer delivers highly competitive performance across various dataset scales and difficulty levels.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Convolution-enhanced dual-stream mixed transformer for 3D human pose estimation in video

  • Shuaijie Pan,
  • Yu Zhi,
  • Shaobo Wang,
  • Ang Chen

摘要

Transformer-based methods have recently made significant advances in 3D human pose estimation, demonstrating excellent performance in global feature extraction and long-range dependency modeling. However, these approaches still face limitations in refining local features and capturing local dependencies. To address this issue, we propose a novel convolution-enhanced dual-stream Transformer framework (DMFormer) designed to model human dependencies with multiple levels of granularity. This increases estimation accuracy through richer pose representations and more accurate motion correlation capture between keypoints. Specifically, DMFormer consists of three main components: (1) a Joint Transformer Encoder, which models spatiotemporal dependencies at the joint level; (2) a Part Transformer Encoder, which focuses on extracting spatiotemporal features at the part level, by leveraging convolutional operations to capture local details and model both global and local dependencies; and (3) a feature fusion layer which integrates multigranularity information to enrich and unify the feature representations. Extensive experiments conducted on two widely used 3D human pose estimation benchmarks, Human3.6M and MPI-INF-3DHP, demonstrate that DMFormer delivers highly competitive performance across various dataset scales and difficulty levels.