Automatic pronunciation assessment (APA) is crucial in computer-assisted pronunciation training (CAPT). Traditional methods typically only extract superficial features at the language level, resulting in excessive redundant information and insufficient extraction of feature correlations at different granularity levels. To address this, we propose the Multi-Task Multi-Level Deep Fusion Model (2MD), which includes a Multi-Granularity Multi-Feature Fusion Module (MMFM) to extract and fuse previously overlooked audio contextual information, and a Convolutional and Gated Combination Deep Feature Extraction and Separation Module (CGA) to deeply process fine-grained features at various language levels, while preserving correlations between different granularities. The effectiveness of our method has been validated on the SpeechOcean762 dataset.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Automatic Speech Evaluation Method Leveraging Deep Feature Fusion

  • Xudong Pang,
  • Wenwen Lu,
  • Silajiaihemaiti Ruzemaimaiti,
  • Aishan Wumaier

摘要

Automatic pronunciation assessment (APA) is crucial in computer-assisted pronunciation training (CAPT). Traditional methods typically only extract superficial features at the language level, resulting in excessive redundant information and insufficient extraction of feature correlations at different granularity levels. To address this, we propose the Multi-Task Multi-Level Deep Fusion Model (2MD), which includes a Multi-Granularity Multi-Feature Fusion Module (MMFM) to extract and fuse previously overlooked audio contextual information, and a Convolutional and Gated Combination Deep Feature Extraction and Separation Module (CGA) to deeply process fine-grained features at various language levels, while preserving correlations between different granularities. The effectiveness of our method has been validated on the SpeechOcean762 dataset.