Evaluating Vision Transformer Variants and Hybrid Quantum-Classical Models for Face Forgery Detection
摘要
Face forgery is becoming an ever more convincing threat to our digital media landscape, thanks to rapid improvements in generative algorithms. In this study, we aim to evaluate the effectiveness of both classical and hybrid quantum-classical transformer-based models in detecting such manipulations. We experimented with four lightweight, pretrained transformer architectures—DeiT, LeViT, MobileViT-XXS, and TinyViT—as well as three hybrid quantum models where quantum layers are integrated into classical backbones: MobileViT-XXS + Quantum, Swin-RY, and Swin-RY + RX + RZ. All models were fine-tuned (or partially trained) using frozen transformer backbones in hybrid models, on a balanced dataset of 140,000 face images—70,000 authentic portraits from NVIDIA’s Flickr-Faces-HQ (FFHQ) and 70,000 synthetic images generated using StyleGAN. Quantum circuits used RY, RX, and RZ rotation gates for angle encoding, along with CNOT gates for qubit entanglement, enabling them to capture non-linear feature interactions. Performance was evaluated using test accuracy, precision, recall, F1-score, confusion matrices, and learning curve analysis under varying hyperparameter and quantum circuit configuration. In the classical group, MobileViT-XXS hit 99.88% accuracy, with TinyViT, LeViT, and DeiT right behind. In the quantum group, Swin-RY + RX + RZ led at 97.42%, edging out both Swin-RY and MobileViT-XXS + Quantum. This suggests that adding a quantum layer helps spot the smallest signs of tampering—things that purely classical models might miss. Combining quantum steps with lightweight vision transformers could give us powerful deepfake detectors that don’t need a lot of computing power. Limitations include reliance on quantum circuit simulation, the use of synthetic deepfakes, and shallow circuit depth. Future work will explore testing on real quantum hardware, adversarial attacks, and multimodal forgery datasets.