Convolution-enhanced dual-stream mixed transformer for 3D human pose estimation in video
摘要
Transformer-based methods have recently made significant advances in 3D human pose estimation, demonstrating excellent performance in global feature extraction and long-range dependency modeling. However, these approaches still face limitations in refining local features and capturing local dependencies. To address this issue, we propose a novel convolution-enhanced dual-stream Transformer framework (DMFormer) designed to model human dependencies with multiple levels of granularity. This increases estimation accuracy through richer pose representations and more accurate motion correlation capture between keypoints. Specifically, DMFormer consists of three main components: (1) a Joint Transformer Encoder, which models spatiotemporal dependencies at the joint level; (2) a Part Transformer Encoder, which focuses on extracting spatiotemporal features at the part level, by leveraging convolutional operations to capture local details and model both global and local dependencies; and (3) a feature fusion layer which integrates multigranularity information to enrich and unify the feature representations. Extensive experiments conducted on two widely used 3D human pose estimation benchmarks, Human3.6M and MPI-INF-3DHP, demonstrate that DMFormer delivers highly competitive performance across various dataset scales and difficulty levels.