It is well known that there is a high correlation between head pose and landmark locations. Some landmark-based head pose estimation methods rely on a 3D face model and alignment algorithms to infer the head pose from landmarks. Due to the nature of the alignment algorithm, the landmark position error will result in head pose error. Some methods regress the head pose from landmarks without the 3D face model. However, the potential relationship between landmarks and head pose is overlooked. To address these issues, we propose a novel transformer architecture, Landmark-Guided head pose estimation (LGFormer), which exploits the landmark relations using the attention mechanism of the transformer and iteratively updates the landmark tokens and refines landmark positions. The head pose is directly regressed from the landmarks without the 3D face model. Experimental results show that our model performs better than existing landmark regression methods and achieves state-of-the-art performance on the AFLW-2000 dataset.