Recent advancements in neural text-to-speech (TTS) have enabled near-human naturalness, yet achieving expressive and controllable emotional speech remains a challenge. Current models struggle with prosodic control due to label dependency, style entanglement, and limited granularity in emotion manipulation. To address these limitations, we propose a novel methodology that integrates unsupervised clustering with multi-modal representations, combining neural style embeddings from StyleTTS 2 with acoustic-prosodic features extracted via ‘librosa’ and ‘emotion2vec’. By leveraging a hierarchical clustering approach using Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN), we achieve improved separation of emotional styles without requiring predefined labels. Experiments on the EmoV-DB and Hi-Fi TTS datasets confirm that our approach effectively captures nuanced emotional variations in speech synthesis, leading to an 200% increase in overall clusters, a reduction of 21.4% of unclustered data samples and an 25.14% increase of homogeneous cluster groups compared to the baseline.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Emotional Speech Synthesis Approach Using Prosody-Based Clustering

  • Arnas Radzevičius,
  • Žygimantas Girdauskas,
  • Rokas Sabaitis,
  • Aistis Raudys

摘要

Recent advancements in neural text-to-speech (TTS) have enabled near-human naturalness, yet achieving expressive and controllable emotional speech remains a challenge. Current models struggle with prosodic control due to label dependency, style entanglement, and limited granularity in emotion manipulation. To address these limitations, we propose a novel methodology that integrates unsupervised clustering with multi-modal representations, combining neural style embeddings from StyleTTS 2 with acoustic-prosodic features extracted via ‘librosa’ and ‘emotion2vec’. By leveraging a hierarchical clustering approach using Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN), we achieve improved separation of emotional styles without requiring predefined labels. Experiments on the EmoV-DB and Hi-Fi TTS datasets confirm that our approach effectively captures nuanced emotional variations in speech synthesis, leading to an 200% increase in overall clusters, a reduction of 21.4% of unclustered data samples and an 25.14% increase of homogeneous cluster groups compared to the baseline.