Cascaded Cross-Attention Vision Transformers with Wavelet-Based Encodings for DeepFake Detection
摘要
DeepFakes have profoundly reshaped the digital landscape through the generation of highly realistic synthetic imagery, posing major obstacles for reliable fake-media detection. In this work, we present Bi-Scalar ViT, a novel architecture that integrates Bi-Scalar Wavelet Encodings with Cascaded Cross-Attention Vision Transformers (ViTs) for DeepFake video detection. The core idea is to exploit two distinct levels of Discrete Wavelet Transform (DWT)–based positional encodings. At the first level, termed the F-Node (Fine Node), we employ the Haar Wavelet Transform to extract high-frequency sub-bands (LH, HL, HH), emphasizing fine-grained details. The second level, denoted as the C-Node (Coarse Node), also models high-frequency components (LH, HL, HH), but utilizes the Daubechies Wavelet Transform, thereby providing a more balanced characterization of local and global patterns. By jointly leveraging these complementary high-frequency representations within a cascaded cross-attention framework, Bi-Scalar_ViT attains both high accuracy and computational efficiency, surpassing current state-of-the-art approaches. We validate the proposed method on three widely used benchmarks: DFD, DFDC, and FF++. Bi-Scalar_ViT achieves 99.5% accuracy on DFD, 97.8% on DFDC, and 98.5% on FF++, with corresponding AUC-ROC scores of 0.998, 0.996, and 0.997. Further experiments on the Celeb-DF dataset confirm the robustness and reliability of the model. Moreover, the system is capable of processing 30 frames per second on a single NVIDIA V100 GPU, highlighting its practicality for real-time or near–real-time deployment. The central contribution of Bi-Scalar_ViT lies in its effective exploitation of these two key frequency-level encodings, enabling accurate and rapid DeepFake detection. Consequently, the framework represents a significant advancement and a powerful tool in countering DeepFake-based media manipulation.