As deepfake technology advances, achieving robust and generalizable detection methods is critical for identifying forgeries across diverse datasets and post-processed manipulations. This work introduces an enhanced GAN-based framework for anomaly detection, utilizing an Attention-Based Refinement Module (ABRM) within the decoder to improve pixel restoration and enhance focus on forgery-prone areas. The ABRM applies selective attention to crucial feature regions, enabling the model to better capture subtle discrepancies between authentic and manipulated data. Additionally, a Criss-Cross Attention (CCA) mechanism in the encoder strengthens spatial dependencies by analyzing pixel-level relationships, producing rich, representative encodings of real data. To increase detection accuracy further, a latent space encoding discriminator is used to verify reconstructed images against the original feature distributions, creating a comprehensive structure for anomaly detection. Training solely on authentic data, the model effectively distinguishes fakes as anomalies based on reconstruction error, without requiring labelled fake samples. Experiments across multiple deepfake datasets, including internet-sourced videos, reveal that this approach achieves superior accuracy and robustness, with ABRM significantly enhancing the model’s sensitivity to fine-grained forgery artifacts and setting a new standard for cross-dataset generalization in deepfake detection.