Voice Synthesis is an Complex field on which Researchers always work. Over the time different advances are been observed in this field for reducing time, complexity, resources required. This research presents a comprehensive voice cloning system that combines state-of-the-art deep learning architectures to achieve high-fidelity speech synthesis with minimal training data. Our approach integrates Tacotron2 for mel-spectrogram prediction, HiFi-GAN as a neural vocoder, and Whisper for improved speech recognition and transcription preprocessing. The system demonstrates remarkable capability in capturing speaker-specific voice characteristics, prosody, and emotional inflections while maintaining natural-sounding speech output. Furthermore, the inclusion of Whisper for preprocessing has improved the system’s robustness to noisy input audio and enhanced phonetic alignment accuracy, resulting in clearer articulation and more consistent prosody in synthesized speech. Our findings suggest that this integrated architecture represents a promising direction for personalized speech synthesis applications across domains including assistive technologies, content creation, and human-computer interaction systems.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Real Time Voice Cloning Using Enhanced TTS and Vocoder

  • Deepika Borgaonkar,
  • M. Hemanth,
  • P. Abhishek,
  • K. Ravi Chandra,
  • P. Srijith

摘要

Voice Synthesis is an Complex field on which Researchers always work. Over the time different advances are been observed in this field for reducing time, complexity, resources required. This research presents a comprehensive voice cloning system that combines state-of-the-art deep learning architectures to achieve high-fidelity speech synthesis with minimal training data. Our approach integrates Tacotron2 for mel-spectrogram prediction, HiFi-GAN as a neural vocoder, and Whisper for improved speech recognition and transcription preprocessing. The system demonstrates remarkable capability in capturing speaker-specific voice characteristics, prosody, and emotional inflections while maintaining natural-sounding speech output. Furthermore, the inclusion of Whisper for preprocessing has improved the system’s robustness to noisy input audio and enhanced phonetic alignment accuracy, resulting in clearer articulation and more consistent prosody in synthesized speech. Our findings suggest that this integrated architecture represents a promising direction for personalized speech synthesis applications across domains including assistive technologies, content creation, and human-computer interaction systems.