Deepfake detection remains a challenging task due to the diversity of forgery techniques and the distributional shifts between training and testing data. Existing methods, often framed as binary classification tasks, struggle with generalization, particularly in real-world scenarios involving unknown forgery types. To address this challenge, we propose DFMoE, a novel deepfake detection framework based on a Heterogeneous Mixture of Experts (HMoE) model. Our approach incorporates dynamic gating networks that adaptively select expert networks of varying capacities and scales according to the input characteristics, enabling precise identification of different types of forgeries. Leveraging a pre-trained face recognition model for multi-scale feature extraction, DFMoE combines expert specialization with adaptive data augmentation to enhance the detection of both known and unknown deepfake types. Experimental results show that our method significantly improves detection accuracy and robustness, offering a highly effective solution for deepfake detection across diverse scenarios.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A New Heterogeneous Mixture of Experts Model for Deepfake Detection

  • Qichang Wang,
  • Ruixia Liu

摘要

Deepfake detection remains a challenging task due to the diversity of forgery techniques and the distributional shifts between training and testing data. Existing methods, often framed as binary classification tasks, struggle with generalization, particularly in real-world scenarios involving unknown forgery types. To address this challenge, we propose DFMoE, a novel deepfake detection framework based on a Heterogeneous Mixture of Experts (HMoE) model. Our approach incorporates dynamic gating networks that adaptively select expert networks of varying capacities and scales according to the input characteristics, enabling precise identification of different types of forgeries. Leveraging a pre-trained face recognition model for multi-scale feature extraction, DFMoE combines expert specialization with adaptive data augmentation to enhance the detection of both known and unknown deepfake types. Experimental results show that our method significantly improves detection accuracy and robustness, offering a highly effective solution for deepfake detection across diverse scenarios.