The rapid advancement of AIGC technologies has enabled the widespread creation of realistic deepfake images, posing significant challenges for laypeople to recognize fake images. In high-stakes scenarios such as media forensics, convincing and interpretable explanations of deepfake detectors are crucial for human decision-making. However, existing methods primarily provide binary real-or-fake predictions, and their visual explanations–such as CAM saliency maps or forgery localization–are typically coarse-grained and fail to indicate which artifacts should be inspected. Recent multimodal large language model (MLLM)-based efforts have explored textual explanations, yet the difficulty of grounding descriptions in images and the risk of contradictory outputs undermine their reliability. To address these challenges, we present FakeArti, a new dataset containing 1,414 high-quality deepfake images with detailed artifact-level explanations and 4,170 pixel-level artifact masks, establishing a benchmark for deepfake explanation evaluation. Furthermore, we propose Attentive Deepfake Artifact Dissection (ADAD), an interpretable detection framework that generates visual grounding of artifacts and textual explanations, offering explicit cues on what artifacts appear and where they are located. ADAD disentangles forgery artifacts from semantic content and enhances multimodal alignment, thereby bridging the gap between model decisions and human reasoning. Extensive experiments demonstrate that ADAD not only achieves state-of-the-art performance in explanation generation and deepfake detection but also exhibits superior generalization ability. This work highlights how incorporating human-perceptible visual artifacts into deepfake detection facilitates more trustworthy and human-centered deepfake forensics.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Dissecting Deepfake Artifacts via Multimodal Explanations

  • Yannan Bai,
  • Danding Wang,
  • Sheng Tang,
  • Juan Cao,
  • Jintao Li

摘要

The rapid advancement of AIGC technologies has enabled the widespread creation of realistic deepfake images, posing significant challenges for laypeople to recognize fake images. In high-stakes scenarios such as media forensics, convincing and interpretable explanations of deepfake detectors are crucial for human decision-making. However, existing methods primarily provide binary real-or-fake predictions, and their visual explanations–such as CAM saliency maps or forgery localization–are typically coarse-grained and fail to indicate which artifacts should be inspected. Recent multimodal large language model (MLLM)-based efforts have explored textual explanations, yet the difficulty of grounding descriptions in images and the risk of contradictory outputs undermine their reliability. To address these challenges, we present FakeArti, a new dataset containing 1,414 high-quality deepfake images with detailed artifact-level explanations and 4,170 pixel-level artifact masks, establishing a benchmark for deepfake explanation evaluation. Furthermore, we propose Attentive Deepfake Artifact Dissection (ADAD), an interpretable detection framework that generates visual grounding of artifacts and textual explanations, offering explicit cues on what artifacts appear and where they are located. ADAD disentangles forgery artifacts from semantic content and enhances multimodal alignment, thereby bridging the gap between model decisions and human reasoning. Extensive experiments demonstrate that ADAD not only achieves state-of-the-art performance in explanation generation and deepfake detection but also exhibits superior generalization ability. This work highlights how incorporating human-perceptible visual artifacts into deepfake detection facilitates more trustworthy and human-centered deepfake forensics.