Recent advancements in deep learning have improved the quality and naturalness of SS. However, these improvements were achieved at the cost of increased model complexity. This work explored and tried to build a scalable framework by proposing a 0.02M parameter vocoder architecture to serve low-resource Indic languages, particularly Bengali and Maithili. We systematically scaled down a 112M architecture, with five parameters: 6M, 1M, 0.5M, 0.1M, and 0.02M parameters, to analyse the trade-off between model compactness and synthesis quality. We used subjective (e.g., MOS, NISQA) and objective metrics (e.g., PESQ, STOI, SNR, MCD, MSD, FAD, PCC). Quantitatively, the LaghuVani-V1 (6M) achieves 4.14 and 2.80 on the 5-scale MOS for Bengali and Maithili, respectively. The compact model LaghuVani-V5 (0.02M, with parameter reduction of 7000 \(\times \) ) achieved the remarkable faster RTF, that is 388 \(\times \) faster on an NVIDIA GTX 1080 8GB GPU with notable MOS of the range of 1.3-1.8 after 1 \(\times \) \(10^{5}\) after (batch size of 8) steps only. Furthermore, we conducted the human-centric evaluation instead of relying on the MOS to assess “How clearly tiny vocoders can articulate speech?”. We found that, for models below 1M, the noise power is greater than the speech power. So it is difficult to hear and recognise the words.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

LaghuVani: How Clearly Can Tiny Vocoders Speak Bengali and Maithili?

  • Kaustubh S. Wade,
  • Ravindrakumar M. Purohit,
  • Hemant A. Patil

摘要

Recent advancements in deep learning have improved the quality and naturalness of SS. However, these improvements were achieved at the cost of increased model complexity. This work explored and tried to build a scalable framework by proposing a 0.02M parameter vocoder architecture to serve low-resource Indic languages, particularly Bengali and Maithili. We systematically scaled down a 112M architecture, with five parameters: 6M, 1M, 0.5M, 0.1M, and 0.02M parameters, to analyse the trade-off between model compactness and synthesis quality. We used subjective (e.g., MOS, NISQA) and objective metrics (e.g., PESQ, STOI, SNR, MCD, MSD, FAD, PCC). Quantitatively, the LaghuVani-V1 (6M) achieves 4.14 and 2.80 on the 5-scale MOS for Bengali and Maithili, respectively. The compact model LaghuVani-V5 (0.02M, with parameter reduction of 7000 \(\times \) ) achieved the remarkable faster RTF, that is 388 \(\times \) faster on an NVIDIA GTX 1080 8GB GPU with notable MOS of the range of 1.3-1.8 after 1 \(\times \) \(10^{5}\) after (batch size of 8) steps only. Furthermore, we conducted the human-centric evaluation instead of relying on the MOS to assess “How clearly tiny vocoders can articulate speech?”. We found that, for models below 1M, the noise power is greater than the speech power. So it is difficult to hear and recognise the words.