Advancements in AI-generated content (AIGC) have made it increasingly difficult to distinguish forgeries in audio and video. Particularly, Deepfake techniques exacerbate this issue by manipulating media within specific time segments, significantly altering the media’s message and complicating detection. Current Deepfake detection methods primarily focus on single modalities, neglecting the potential of cross-modal correspondence for identifying forgeries, or remain confined to binary classification tasks without the ability to localize forgeries in the temporal dimension. We observe that the differences between real and fake data pairs offer valuable cues for temporal localization, yet remain under-explored. To address these gaps, we propose a multimodal temporal forgery localization method that leverages cross-modal correspondence to model inter-modal relationships and exploits fine-grained genuine-forged differences to extract discriminative features. Experiments on the LAV-DF and AV-Deepfake1M datasets demonstrate that our method achieves state-of-the-art results, validating its effectiveness.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Tell Fake from Real: Temporal Forgery Localization with Fake-Real Guided Cross-Modal-Reconstruction

  • Xu Yang,
  • Yanhui Zhou,
  • Yaowen Xu,
  • Zhaofan Zou,
  • Chenyang Ge,
  • Zhixiang He

摘要

Advancements in AI-generated content (AIGC) have made it increasingly difficult to distinguish forgeries in audio and video. Particularly, Deepfake techniques exacerbate this issue by manipulating media within specific time segments, significantly altering the media’s message and complicating detection. Current Deepfake detection methods primarily focus on single modalities, neglecting the potential of cross-modal correspondence for identifying forgeries, or remain confined to binary classification tasks without the ability to localize forgeries in the temporal dimension. We observe that the differences between real and fake data pairs offer valuable cues for temporal localization, yet remain under-explored. To address these gaps, we propose a multimodal temporal forgery localization method that leverages cross-modal correspondence to model inter-modal relationships and exploits fine-grained genuine-forged differences to extract discriminative features. Experiments on the LAV-DF and AV-Deepfake1M datasets demonstrate that our method achieves state-of-the-art results, validating its effectiveness.