Deepfakes are fake images and videos created levraging advanced AI techniques like Generative Adversarial Networks (GANs), Vision Transformers (ViTs), and Autoencoders. These fake media can be difficult to detect as they closely resemble real ones. This paper proposes a new method called Multi-Fusion and Residual Learning in Deep Convolutional Neural Network to identify deepfake images more accurately. The proposed model combines three powerful deep learning models—Xception, Swin Transformer, and EfficientNetV2-B2 to extract important features from images. By using a technique called transfer learning, we freeze and modify certain layers to improve performance. A fusion residual block then processes these features, helping the model detect fake images more effectively. The results clearly depict that the proposed model easily outperforms traditional deep learning models, achieving higher accuracy. This approach can be useful in social media content monitoring, crime investigations, and medical image verification, helping to protect digital content from manipulation. The proposed Multifusion architecture attained a training accuracy of 99.5%, validation accuracy of 98.36%, test accuracy of 96.16%, F1-Score of 96.16% and ROC-AUC of 99.41%.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Deepfake Detection with Multi-Fusion and Residual Learning in Deep CNNs

  • Ayush Shaurya Jha,
  • Tanishq Rajoria,
  • Bam Bahadur Sinha,
  • Alongbar Wary

摘要

Deepfakes are fake images and videos created levraging advanced AI techniques like Generative Adversarial Networks (GANs), Vision Transformers (ViTs), and Autoencoders. These fake media can be difficult to detect as they closely resemble real ones. This paper proposes a new method called Multi-Fusion and Residual Learning in Deep Convolutional Neural Network to identify deepfake images more accurately. The proposed model combines three powerful deep learning models—Xception, Swin Transformer, and EfficientNetV2-B2 to extract important features from images. By using a technique called transfer learning, we freeze and modify certain layers to improve performance. A fusion residual block then processes these features, helping the model detect fake images more effectively. The results clearly depict that the proposed model easily outperforms traditional deep learning models, achieving higher accuracy. This approach can be useful in social media content monitoring, crime investigations, and medical image verification, helping to protect digital content from manipulation. The proposed Multifusion architecture attained a training accuracy of 99.5%, validation accuracy of 98.36%, test accuracy of 96.16%, F1-Score of 96.16% and ROC-AUC of 99.41%.