In recent years, End-to-End Neural Diarization (EEND) has emerged as a prominent research direction in the field of speech processing, owing to its ability to jointly model speaker activity detection and speaker classification within a unified framework. However, most existing EEND models rely on Transformer-based encoders, which often encounter performance bottlenecks when dealing with complex conditions such as speech overlaps, rapid speaker transitions, and fine-grained boundary modeling. To address these challenges, we proposes a novel end-to-end neural diarization system named Structurally Enhanced EEND (SE-EEND), based on a MULTI-CONVFORMER encoder architecture. This architecture incorporates multi-scale convolutional kernels and gating mechanisms, enhancing the model’s capacity to capture both local and global dependencies at varying temporal resolutions. Furthermore, a Bi-Path Residual Block (BPRB) is introduced at the encoder front-end, which integrates standard and dilated convolutions in parallel to structurally enrich the input features. Extensive evaluations across multiple publicly available datasets demonstrate that SE-EEND consistently outperforms mainstream EEND systems in terms of Diarization Error Rate (DER). Notably, on real-world meeting corpora such as AISHELL-4, AliMeeting, AMI, DIHARD-III and RAMC, SE-EEND achieves state-of-the-art performance.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

SE-EEND: A Structurally Enhanced End-to-End Neural Diarization System

  • Penghao Ma,
  • Guangcun Wei,
  • Chuike Kong,
  • Shuo Li,
  • Jianfeng Fang

摘要

In recent years, End-to-End Neural Diarization (EEND) has emerged as a prominent research direction in the field of speech processing, owing to its ability to jointly model speaker activity detection and speaker classification within a unified framework. However, most existing EEND models rely on Transformer-based encoders, which often encounter performance bottlenecks when dealing with complex conditions such as speech overlaps, rapid speaker transitions, and fine-grained boundary modeling. To address these challenges, we proposes a novel end-to-end neural diarization system named Structurally Enhanced EEND (SE-EEND), based on a MULTI-CONVFORMER encoder architecture. This architecture incorporates multi-scale convolutional kernels and gating mechanisms, enhancing the model’s capacity to capture both local and global dependencies at varying temporal resolutions. Furthermore, a Bi-Path Residual Block (BPRB) is introduced at the encoder front-end, which integrates standard and dilated convolutions in parallel to structurally enrich the input features. Extensive evaluations across multiple publicly available datasets demonstrate that SE-EEND consistently outperforms mainstream EEND systems in terms of Diarization Error Rate (DER). Notably, on real-world meeting corpora such as AISHELL-4, AliMeeting, AMI, DIHARD-III and RAMC, SE-EEND achieves state-of-the-art performance.