Non-rigid physiological motion during Cone-Beam Computed Tomography (CBCT) acquisitions remains a significant clinical challenge. To address this, we introduce the Latent Spatio-Temporal Transformer (LSTT), an end-to-end framework designed to directly correct motion artifacts from projection data. Our entirely image-based approach requires only the original CBCT projections and imaging geometry, eliminating the necessity for respiratory or ECG gating and external monitoring devices. The LSTT architecture integrates a VQ-VAE to tokenize projections into a robust latent space, a temporal Transformer to capture global motion dynamics, and a decoder to produce explicit 2D displacement fields. Central to our framework is a differentiable Feldkamp-Davis-Kress (FDK) reconstruction layer, which enables true end-to-end training by optimizing the objective function on the final reconstructed volume. This approach compels the network to learn a physically meaningful policy for non-rigid motion, explicitly tailored for high-fidelity volumetric reconstruction. We validate our framework using a realistic respiratory motion phantom, demonstrating significant improvements over the standard clinical baseline in both artifact suppression and structural preservation.