Scholarlink

<p>With the rapid advancements in generative adversarial networks (GANs), neural vocoders have emerged as critical components for synthesizing intelligible speech. The rise of fake audio poses significant challenges and risks to national security due to malicious abuse. Although countermeasures have been proposed to detect deepfakes, attributing audio to specific vocoder architectures remains a challenging task. Existing approaches that directly input handcrafted features into sophisticated deep neural networks (DNNs) tend to neglect the misguidance of content-relevant features, which leads to poor generalization and efficacy. In this paper, we propose a novel framework that focuses on disentangling the vocoder fingerprint from audio to identify fake audio. To this end, we introduce an audio reconstructor based on the U-Net architecture that minimizes the preservation of the content-relevant features of the original audio. The residual between the raw and reconstructed latent vectors is then calculated to eliminate content-relevant features. The residual is finally fed into a classifier to determine the vocoder’s architecture. The extensive experiments demonstrate the effectiveness of our proposed method in attributing fake audio in various cross-test setups on large-scale datasets. Additionally, we apply our approach to binary fake audio detection and observe its remarkable generalizability even with unseen vocoders.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Understanding and leveraging vocoder fingerprints for synthetic speech attribution

  • 期刊论文

  • 发表日期：

    2025年4月21日

查看全文
  • Jianpeng Ke,
  • Lina Wang

摘要

With the rapid advancements in generative adversarial networks (GANs), neural vocoders have emerged as critical components for synthesizing intelligible speech. The rise of fake audio poses significant challenges and risks to national security due to malicious abuse. Although countermeasures have been proposed to detect deepfakes, attributing audio to specific vocoder architectures remains a challenging task. Existing approaches that directly input handcrafted features into sophisticated deep neural networks (DNNs) tend to neglect the misguidance of content-relevant features, which leads to poor generalization and efficacy. In this paper, we propose a novel framework that focuses on disentangling the vocoder fingerprint from audio to identify fake audio. To this end, we introduce an audio reconstructor based on the U-Net architecture that minimizes the preservation of the content-relevant features of the original audio. The residual between the raw and reconstructed latent vectors is then calculated to eliminate content-relevant features. The residual is finally fed into a classifier to determine the vocoder’s architecture. The extensive experiments demonstrate the effectiveness of our proposed method in attributing fake audio in various cross-test setups on large-scale datasets. Additionally, we apply our approach to binary fake audio detection and observe its remarkable generalizability even with unseen vocoders.