Source Vendor Tracing of Audio Deepfakes
摘要
With the growing prevalence of synthetic speech in both benign and malicious applications, the ability to trace the origin of generated audio has become increasingly important. In this paper, we propose the vendor tracing task, that is, the source tracing from the perspective of public speech generation systems, offered by various vendors. Modern speech services such as ElevenLabs and PlayHT offer highly accessible and advanced voice cloning features, making them potential tools for generating deepfakes by fraudsters. Given their possible use in malicious scenarios, identifying the system used to generate fake audio is valuable to forensic experts. A dataset of fake speech from various vendors was collected to conduct the investigation. The Audio SSL Encoder-based model was evaluated on a vendor tracing task, while encoder depth, encoder architecture, fine-tuning strategies, and the influence of pretraining the encoder on the audio deepfake detection task were investigated, in open-set and closed-set classification scenarios. Results show that audio deepfake detection models need to be unfrozen and fine-tuned for the vendor tracing task, and the open-set training regime for the model is generally better than the closed-set one, but an application of additional calibration is required to trade-off between detection quality of target vendors and out-of-domain vendors. We obtain 95% in terms of Macro Accuracy for the closed-set task and 93% for the open-set task.