<p>Due to the lack of effective information exchange and updating mechanisms between speaker clue and extraction modules, coupled with inadequate feature extraction and representation capabilities, most existing target speaker extraction (TSE) models are difficult to promptly adjust model parameters according to dynamically changing mixed speech features, and their robustness and generalization are significantly limited in practical applications. To address these issues, this study proposes an improved TSE method with adaptive information exchange and updating based on emphasized channel attention, propagation and aggregation time delay neural network (ECAPA-TDNN) and gated convolutional recurrent network (GCRN). First, a TSE framework based on ECAPA-TDNN-GCRN is constructed using two simultaneously trained modules: ECAPA-TDNN-based speaker clue module and GCRN-based speaker extraction module. Then, an attention fusion layer between the speaker clue and extraction modules is designed, enhancing the model’s ability to integrate and utilize different scale feature information. Additionally, a feedback mechanism based on multi-task training is implemented to provide a strategy for the speaker extraction module that adaptively regulates and updates the parameters of the clue module. Finally, multi-dimensional data augmentation is employed to simulate complex auditory scenes, aiming to improve model robustness, and batch augmentation is integrated into the model training process to participate in model parameter gradient updates, thereby enabling faster training and better generalization. The proposed method supports parallel and scalable training, leveraging GPU-accelerated platforms to enable efficient multi-task optimization and low-latency inference. Multiple experiments on the Librispeech dataset demonstrate that our method can significantly improve the TSE performance in various complex scenarios.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Accurate target speaker extraction method with adaptive information interaction and updating in complex scenarios

  • Yangjie Wei,
  • Ben Niu,
  • Yuqiao Wang,
  • Xiaoli Zhang

摘要

Due to the lack of effective information exchange and updating mechanisms between speaker clue and extraction modules, coupled with inadequate feature extraction and representation capabilities, most existing target speaker extraction (TSE) models are difficult to promptly adjust model parameters according to dynamically changing mixed speech features, and their robustness and generalization are significantly limited in practical applications. To address these issues, this study proposes an improved TSE method with adaptive information exchange and updating based on emphasized channel attention, propagation and aggregation time delay neural network (ECAPA-TDNN) and gated convolutional recurrent network (GCRN). First, a TSE framework based on ECAPA-TDNN-GCRN is constructed using two simultaneously trained modules: ECAPA-TDNN-based speaker clue module and GCRN-based speaker extraction module. Then, an attention fusion layer between the speaker clue and extraction modules is designed, enhancing the model’s ability to integrate and utilize different scale feature information. Additionally, a feedback mechanism based on multi-task training is implemented to provide a strategy for the speaker extraction module that adaptively regulates and updates the parameters of the clue module. Finally, multi-dimensional data augmentation is employed to simulate complex auditory scenes, aiming to improve model robustness, and batch augmentation is integrated into the model training process to participate in model parameter gradient updates, thereby enabling faster training and better generalization. The proposed method supports parallel and scalable training, leveraging GPU-accelerated platforms to enable efficient multi-task optimization and low-latency inference. Multiple experiments on the Librispeech dataset demonstrate that our method can significantly improve the TSE performance in various complex scenarios.