With the growing prevalence of synthetic speech in both benign and malicious applications, the ability to trace the origin of generated audio has become increasingly important. In this paper, we propose the vendor tracing task, that is, the source tracing from the perspective of public speech generation systems, offered by various vendors. Modern speech services such as ElevenLabs and PlayHT offer highly accessible and advanced voice cloning features, making them potential tools for generating deepfakes by fraudsters. Given their possible use in malicious scenarios, identifying the system used to generate fake audio is valuable to forensic experts. A dataset of fake speech from various vendors was collected to conduct the investigation. The Audio SSL Encoder-based model was evaluated on a vendor tracing task, while encoder depth, encoder architecture, fine-tuning strategies, and the influence of pretraining the encoder on the audio deepfake detection task were investigated, in open-set and closed-set classification scenarios. Results show that audio deepfake detection models need to be unfrozen and fine-tuned for the vendor tracing task, and the open-set training regime for the model is generally better than the closed-set one, but an application of additional calibration is required to trade-off between detection quality of target vendors and out-of-domain vendors. We obtain 95% in terms of Macro Accuracy for the closed-set task and 93% for the open-set task.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Source Vendor Tracing of Audio Deepfakes

  • Marina Volkova,
  • Artem Chirkovskiy,
  • Egor Ausev,
  • Ekaterina Shangina

摘要

With the growing prevalence of synthetic speech in both benign and malicious applications, the ability to trace the origin of generated audio has become increasingly important. In this paper, we propose the vendor tracing task, that is, the source tracing from the perspective of public speech generation systems, offered by various vendors. Modern speech services such as ElevenLabs and PlayHT offer highly accessible and advanced voice cloning features, making them potential tools for generating deepfakes by fraudsters. Given their possible use in malicious scenarios, identifying the system used to generate fake audio is valuable to forensic experts. A dataset of fake speech from various vendors was collected to conduct the investigation. The Audio SSL Encoder-based model was evaluated on a vendor tracing task, while encoder depth, encoder architecture, fine-tuning strategies, and the influence of pretraining the encoder on the audio deepfake detection task were investigated, in open-set and closed-set classification scenarios. Results show that audio deepfake detection models need to be unfrozen and fine-tuned for the vendor tracing task, and the open-set training regime for the model is generally better than the closed-set one, but an application of additional calibration is required to trade-off between detection quality of target vendors and out-of-domain vendors. We obtain 95% in terms of Macro Accuracy for the closed-set task and 93% for the open-set task.