The rapid rise of deepfake technology has given rise to deep concerns about the potential of digital media. However, in the case of audio-free datasets like Celeb-DF we also need detection approaches that make use only of the video. This paper introduces a multimodal deepfake detection pipeline composed of visual analysis using EfficientNetB0, post-hoc image appreciation using XAI and forensics such as compression artifacts analysis, metadata anomaly detection and watermark search in order to reveal the signs of fraud with unparalleled granularity. Although this physical technique is superior to traditional methods in terms of accuracy and reliability, it also provides clear visualizations (GradCAM heatmap) for the users to explain how each decision is made. We believe that this framework will add significant complementary security and digital forensics to bring a more resilient and transparent style of deepfake detection.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multimodal Deepfake Detection: Integrating Video Based Analysis, Explainable AI, and Forensic Techniques

  • Ritik Kumar Patra,
  • Anirban Dalui,
  • Shubham Singh,
  • Parth Dwivedi,
  • J. S. Nixon

摘要

The rapid rise of deepfake technology has given rise to deep concerns about the potential of digital media. However, in the case of audio-free datasets like Celeb-DF we also need detection approaches that make use only of the video. This paper introduces a multimodal deepfake detection pipeline composed of visual analysis using EfficientNetB0, post-hoc image appreciation using XAI and forensics such as compression artifacts analysis, metadata anomaly detection and watermark search in order to reveal the signs of fraud with unparalleled granularity. Although this physical technique is superior to traditional methods in terms of accuracy and reliability, it also provides clear visualizations (GradCAM heatmap) for the users to explain how each decision is made. We believe that this framework will add significant complementary security and digital forensics to bring a more resilient and transparent style of deepfake detection.