A Survey on the Integration of Multimodal Techniques for Visual Artifact Detection: Advancing Lip–Speech Synchronization and Facial Recognition
摘要
Deepfake technology has emerged as a double-edged sword enabling creative innovation while posing serious risks to digital security and public trust. Powered by advanced generative models like GANs and diffusion networks, deepfakes have become increasingly difficult to detect. This paper explores the evolution of these techniques with a focus on audio-visual forgeries and their detection. We highlight lip-speech synchronization analysis as a powerful yet underutilized tool, capable of revealing subtle desynchronization artifacts, such as phoneme-lip mismatches and unnatural motion, often missed by traditional visual methods. A comprehensive review of detection strategies across visual, audio-visual, and text-visual modalities is presented. Benchmark datasets (FaceForensics++, DFDC, FakeAVCeleb, WildDeepfake) are examined. Finally, we discuss future research directions. By focusing on lip–speech synchronization inconsistencies, an area where generative models still struggle to achieve perfection, this work identifies promising approaches for enhancing the overall effectiveness of deepfake detection.