Exposing Audio-Visual Forgeries in Frequency Domain
摘要
Recently, the rapid development of deepfake technology attracted strong attention from the community. Some previous work on deepfake detection achieved good results in the frequency domain, which inspires us to combine frequency-domain information with temporal and spatial domains of visual to detect deepfakes. In addition, the audio signal can be represented in the frequency domain, so we can explore multimodal frequency-domain cues by combining audio and visual modalities. In this paper, we propose a Frequency-aware Audio-Visual Deepfake Detection(FAVDD) method. Specifically, we design a Frequency-Temporal-Spatial(FTS) visual encoder that extracts spatial, frequency, and temporal forgery cues and embeds them into visual features to form a unified representation. In addition, we project the audio signal into the frequency domain by Fourier transform and capture the forgery traces, which are later combined with visual features for deepfake detection. The results show that our proposed framework effectively combines multiple cues and achieves good results on three multimodal deepfake datasets.