Head pose estimation involves determining a person’s head orientation relative to a camera by estimating rotation angles around the yaw, pitch, and roll axes. Typically, this task is performed using single RGB images to minimize computational demands and model sizes. While some methods analyze video sequences frame-by-frame for face detection, these often overlook the rich temporal information available in videos that could enhance accuracy. Recurrent neural networks (RNNs) have been used for this purpose but require extensive training and memory. This chapter introduces a novel spatial-temporal aggregation framework for estimating head pose from video sequences. Unlike RNN-based approaches, our method leverages temporal information through a temporal aggregation module and parallel attention to focus on specific features at different times. To address challenges in face detection and background noise, we implement progressive spatial weighting to concentrate on the head region. Additionally, a multistage regression strategy is used to improve robustness and flexibility. Our method demonstrates superior performance compared to existing algorithms on both the BIWI and our own video datasets.