Scholarlink

With the rapid development of AI synthesized voice, generative audio watermarking technology is urgently needed to protect people’s privacy and maintain social stability. Given Wav2vec 2.0’s strong performance in fake-voice detection and based on the need to improve the robustness of current end-to-end audio watermarking against composite attacks, we propose TS-EWM, an active watermarking method that embeds watermarks directly into zero-shot synthetic speech via VALL-E. We use Wav2vec 2.0 for decoding in a multi-branch decoder that fuses ResNet-based Mel-spectrum features, Wav2vec 2.0’s CNN’s low-level phoneme cues, and Transformer’s high-level semantics. A novel cross-attention mechanism treats ResNet outputs as queries and Wav2vec 2.0 features as keys/values, with an adaptive gating module to balance multimodal, multi-scale information. We integrate PESQ and STOI perceptual metrics into the loss and apply orthogonal gradient projection to harmonize audio fidelity and watermark robustness. In both in-domain and cross-domain tests on Chinese and English datasets, TS-EWM significantly improves watermark extraction robustness while preserving high audio quality.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

TS-EWM: An End-to-End Speech Watermarking Scheme Based On Multimodal Multi-branch Decoder And Multi-sensory Orthogonal Projection Gradient Optimization

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Chengde Zhang,
  • Guangcun Wei,
  • Yanhong Long,
  • Wenjing Wang,
  • Kangjin Zuo

摘要

With the rapid development of AI synthesized voice, generative audio watermarking technology is urgently needed to protect people’s privacy and maintain social stability. Given Wav2vec 2.0’s strong performance in fake-voice detection and based on the need to improve the robustness of current end-to-end audio watermarking against composite attacks, we propose TS-EWM, an active watermarking method that embeds watermarks directly into zero-shot synthetic speech via VALL-E. We use Wav2vec 2.0 for decoding in a multi-branch decoder that fuses ResNet-based Mel-spectrum features, Wav2vec 2.0’s CNN’s low-level phoneme cues, and Transformer’s high-level semantics. A novel cross-attention mechanism treats ResNet outputs as queries and Wav2vec 2.0 features as keys/values, with an adaptive gating module to balance multimodal, multi-scale information. We integrate PESQ and STOI perceptual metrics into the loss and apply orthogonal gradient projection to harmonize audio fidelity and watermark robustness. In both in-domain and cross-domain tests on Chinese and English datasets, TS-EWM significantly improves watermark extraction robustness while preserving high audio quality.