KT-STFormer: Parallel Kinematic-Trajectory and Spatial-Temporal Fusion Transformer for 3D Human Pose Estimation
摘要
Recently, Transformer-based methods have achieved remarkable performance in 3D human pose estimation. Nevertheless, the self-attention mechanism of traditional Transformers overlook the global interdependencies among joints as well as the long-term motion patterns inherent to these joints. In this paper, we propose a novel Parallel Kinematic-Trajectory and Spatial-Temporal Fusion Transformer (KT-STFormer), comprising two core modules: the Parallel Kinematic-Trajectory Fusion Transformer (PKTF) and the Parallel Spatial-Temporal Fusion Transformer (PSTF). Specifically, PKTF splits input features along the channel dimension into two branches: one embeds kinematic prior via Kinematics Prior Attention (KPA) for global joint correlation learning through Structure-Aware Spatial MHSA, while the other integrates cross-frame trajectory prior via Trajectory Prior Attention (TPA) and Structure-Aware Temporal MHSA. PSTF similarly splits input features into two channel-wise parts, applies spatial and temporal attention respectively, and models spatiotemporal dependencies by concatenating the attention outputs, which substantially reduces the computational complexity of the model. We further integrate a Structure-enhanced Positional Embedding (SPE) into the KT-STFormer to account for the structure of the human body. Additionally, we introduce Attention with Linear Biases (ALiBi) into the temporal attention mechanism to explicitly model temporal biases. Extensive experiments on three benchmarks (Human3.6M, MPI-INF-3DHP and HumanEva) demonstrate that the proposed KT-STFormer achieves competitive and robust performance compared with state-of-the-art Transformer-based methods for 3D human pose estimation.