Scholarlink

<p>Voice Conversion (VC) is the task of changing the speaker characteristics in a source speech signal to be like the one in a target speech while keeping the linguistic content of the source. VC is widely used in many applications such as speaking assistance systems for people with vocal disorders. Non-parallel many-to-many and non-parallel zero-shot VC are the most challenging versions of the VC task. So many deep learning-based VC models made successful efforts to solve these tasks but they suffer from training instability, over-smoothing, distribution mismatch or the usage of external models which limit the model ability and scalability. In this paper, we propose a VC model that focuses on the non-parallel many-to-many and zero-shot VC tasks. Our main goal in this paper is to solve the VC task using a model which overcomes the mentioned problems and achieves comparable results to the complex models using a simple architecture. The proposed model is based on an Auto-Encoder-based model (AutoVC) while using Adaptive Instance Normalization. The used dataset is the VCTK corpus. Our proposed model is tested in zero-shot and many-to-many VC tasks. Zero-shot VC test is divided into subtests (1 and 2) differentiated in the testing speakers’ identity. Four evaluation metrics are used in each test; Mel-Cepstral Distortion (MCD), d-vector cosine similarity, mean-opinion-score (using MosNet) and Word Error Rate (WER). In many-to-many test, our model achieves MCD of 4.83, d-vector cosine similarity of 0.73, MosNet score of 3.78 and WER of 0.48. In subtest 1 of zero-shot VC test, our model achieves MCD of 4.08, d-vector cosine similarity of 0.66, MosNet score of 3.72 and WER of 0.63. In subtest 2 of zero-shot VC test, our model achieves MCD of 4.60, d-vector cosine similarity of 0.63, MosNet score of 3.89 and WER of 0.66.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Zero-shot non-parallel voice conversion using auto-encoder and adaptive instance normalization

  • 期刊论文

  • 发表日期：

    2025年5月28日

查看全文
  • Yasmin Alaa,
  • Mostafa M. Aref,
  • Marco Alfonse

摘要

Voice Conversion (VC) is the task of changing the speaker characteristics in a source speech signal to be like the one in a target speech while keeping the linguistic content of the source. VC is widely used in many applications such as speaking assistance systems for people with vocal disorders. Non-parallel many-to-many and non-parallel zero-shot VC are the most challenging versions of the VC task. So many deep learning-based VC models made successful efforts to solve these tasks but they suffer from training instability, over-smoothing, distribution mismatch or the usage of external models which limit the model ability and scalability. In this paper, we propose a VC model that focuses on the non-parallel many-to-many and zero-shot VC tasks. Our main goal in this paper is to solve the VC task using a model which overcomes the mentioned problems and achieves comparable results to the complex models using a simple architecture. The proposed model is based on an Auto-Encoder-based model (AutoVC) while using Adaptive Instance Normalization. The used dataset is the VCTK corpus. Our proposed model is tested in zero-shot and many-to-many VC tasks. Zero-shot VC test is divided into subtests (1 and 2) differentiated in the testing speakers’ identity. Four evaluation metrics are used in each test; Mel-Cepstral Distortion (MCD), d-vector cosine similarity, mean-opinion-score (using MosNet) and Word Error Rate (WER). In many-to-many test, our model achieves MCD of 4.83, d-vector cosine similarity of 0.73, MosNet score of 3.78 and WER of 0.48. In subtest 1 of zero-shot VC test, our model achieves MCD of 4.08, d-vector cosine similarity of 0.66, MosNet score of 3.72 and WER of 0.63. In subtest 2 of zero-shot VC test, our model achieves MCD of 4.60, d-vector cosine similarity of 0.63, MosNet score of 3.89 and WER of 0.66.