Spatial-Temporal Pyramid for 3D Human Head Pose Prediction
摘要
The prediction of 3D human head pose aims to forecast future head positions based on an observed sequence, a task increasingly significant in human-computer interaction, virtual reality, and driver monitoring systems. However, due to the numerous possible head movements, existing head pose estimation methods primarily focus on estimation and fall short in providing adequate temporal information for precise predictions. This chapter introduces a spatial-temporal pyramid framework that consists of a temporal pyramid encoder (TPE), a module for aggregating time-step features, and a displacement-based offset generation module, for 3D human head pose prediction. The TPE uses a pyramid structure to extract multi-level spatial-temporal information, merging spatial data from the Transformer encoder with temporal information from the sequential order of frames. The time-step feature aggregation module integrates features based on information density and hierarchically extracts past motion data as prior knowledge to capture motion recurrence. Additionally, the displacement-based offset generation module uses displacement information through a frequency domain process between consecutive frames to produce an offset that refines the raw prediction results. Extensive experiments on the proposed dataset demonstrate that the network significantly outperforms related methods, with a 50% relative improvement for 100 ms predictions and 40% for 300 ms predictions.