<p>Existing time-domain SER methods often rely on fixed time windows, which neither adapt to the dynamic temporal distribution of emotional cues nor balance local emotional details of global temporal context, limiting their recognition performances. To address these issues, a novel Global-Local Fusion Transformer with Adaptive Window, AW-GLFormer for short, is proposed. It comprises three core modules: a pre-trained large model WavLM-Large, a RELEV block, and a WPGL block. The pre-trained large model WavLM-Large extract a temporal token sequence from raw speech signals; the RELEV block quantifies the strength of feature interactions into relevance scores, while also obtaining preliminary features; WPGL block adaptively divides windows based on relevance scores, analyzes subtle emotional changes within the windows to obtain window-level features, and ultimately breaks the window limitations to integrate information from the entire sequence to obtain global features. Experimental results demonstrate that the proposed AW-GLFormer achieves a WA of 72.9% and a UA of 73.8%, respectively, on the IEMOCAP dataset, along with a WF1 of 49.1% on the MELD dataset. These performance values significantly outperform those of existing SER methods. Ablation experiments further validate the effectiveness of each component of AW-GLFormer.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Global-Local Fusion Transformer with Adaptive Window for Speech Emotion Recognition

  • Xin Yang,
  • Heming Huang,
  • Yonghong Fan,
  • Kedi Huang

摘要

Existing time-domain SER methods often rely on fixed time windows, which neither adapt to the dynamic temporal distribution of emotional cues nor balance local emotional details of global temporal context, limiting their recognition performances. To address these issues, a novel Global-Local Fusion Transformer with Adaptive Window, AW-GLFormer for short, is proposed. It comprises three core modules: a pre-trained large model WavLM-Large, a RELEV block, and a WPGL block. The pre-trained large model WavLM-Large extract a temporal token sequence from raw speech signals; the RELEV block quantifies the strength of feature interactions into relevance scores, while also obtaining preliminary features; WPGL block adaptively divides windows based on relevance scores, analyzes subtle emotional changes within the windows to obtain window-level features, and ultimately breaks the window limitations to integrate information from the entire sequence to obtain global features. Experimental results demonstrate that the proposed AW-GLFormer achieves a WA of 72.9% and a UA of 73.8%, respectively, on the IEMOCAP dataset, along with a WF1 of 49.1% on the MELD dataset. These performance values significantly outperform those of existing SER methods. Ablation experiments further validate the effectiveness of each component of AW-GLFormer.