Scholarlink

<p>Text-to-speech (TTS), singing voice synthesis (SVS), and opera singing synthesis (OSS) aim to convert linguistic or musical inputs into intelligible and expressive speech. Although these tasks share a similar generative pipeline, they differ substantially in acoustic characteristics such as pitch range, prosody, and expressiveness. These distinctions pose significant challenges for unified modeling, particularly in learning representations that are both generalizable across tasks and sufficiently expressive for each task’s specific requirements. Existing approaches typically employ separate models for each task, resulting in increased computational overhead and limited cross-task knowledge transfer. To address these challenges, we propose UniVoice, a unified voice synthesis framework designed to jointly model TTS, SVS, and OSS. UniVoice adopts discrete acoustic units as a unified representation across tasks, simplifying the learning objective. To accommodate task-specific variability, we introduce the task-specific adapter with Top-1 routing that enables the acoustic model to specialize through lightweight, learnable adapters, effectively mitigating cross-task interference while preserving parameter efficiency. Furthermore, we incorporate a task-aware postnet that refines unit sequences based on task identity and musical context, enhancing both expressiveness and output quality. The final waveform is synthesized using a unit-based vocoder built on BigVGAN, conditioned on discrete units and pitch information. Experiments on multiple datasets spanning TTS, SVS, and OSS demonstrate that UniVoice significantly outperforms existing single-task and multi-task baselines. Specifically, UniVoice achieves improvements of 0.26, 0.12, and 0.22 in average Mean Opinion Scores (MOS) over the multi-task baseline for TTS, SVS, and OSS, respectively. Moreover, UniVoice achieves these gains with a compact model size of only 49.4M parameters and maintains a fast inference speed, indicating its effectiveness in balancing synthesis quality and computational efficiency.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

UniVoice: a unified framework for text-to-speech, singing voice synthesis, and opera singing synthesis

  • 期刊论文

  • 发表日期：

    2026年2月20日

查看全文
  • Yue Zhou,
  • Peng Bai,
  • Xiaodong Shi

摘要

Text-to-speech (TTS), singing voice synthesis (SVS), and opera singing synthesis (OSS) aim to convert linguistic or musical inputs into intelligible and expressive speech. Although these tasks share a similar generative pipeline, they differ substantially in acoustic characteristics such as pitch range, prosody, and expressiveness. These distinctions pose significant challenges for unified modeling, particularly in learning representations that are both generalizable across tasks and sufficiently expressive for each task’s specific requirements. Existing approaches typically employ separate models for each task, resulting in increased computational overhead and limited cross-task knowledge transfer. To address these challenges, we propose UniVoice, a unified voice synthesis framework designed to jointly model TTS, SVS, and OSS. UniVoice adopts discrete acoustic units as a unified representation across tasks, simplifying the learning objective. To accommodate task-specific variability, we introduce the task-specific adapter with Top-1 routing that enables the acoustic model to specialize through lightweight, learnable adapters, effectively mitigating cross-task interference while preserving parameter efficiency. Furthermore, we incorporate a task-aware postnet that refines unit sequences based on task identity and musical context, enhancing both expressiveness and output quality. The final waveform is synthesized using a unit-based vocoder built on BigVGAN, conditioned on discrete units and pitch information. Experiments on multiple datasets spanning TTS, SVS, and OSS demonstrate that UniVoice significantly outperforms existing single-task and multi-task baselines. Specifically, UniVoice achieves improvements of 0.26, 0.12, and 0.22 in average Mean Opinion Scores (MOS) over the multi-task baseline for TTS, SVS, and OSS, respectively. Moreover, UniVoice achieves these gains with a compact model size of only 49.4M parameters and maintains a fast inference speed, indicating its effectiveness in balancing synthesis quality and computational efficiency.