Automatic English-Speaking Fluency Scoring System Based on Automatic Speech Assessment
摘要
With the increasing requirement for language learning and evaluation tools, it is necessary to develop an automated speech assessment system. However, the existing systems rarely deliver a clear score for speech fluency and pronunciation, which is lower than human perception. The goal of this research is to develop an Automatic English-Speaking Fluency Scoring System (AESFSS) to assess the fluency of English speakers using multiple speech variables. This approach utilizes deep learning (DL) models and a multidimensional approach to extract critical speech parameters, including speech rate, pause duration, pitch change, and pronunciation accuracy. The dataset comprises voices in the English language, recorded from various speakers. These samples vary in speaking styles, pronunciation, and fluency levels. Data preprocessing uses the Wiener Filter to reduce noise and feature extraction by Mel-frequency Cepstral Coefficients (MFCCs) for measuring the spectral properties of speech. A scoring algorithm with a dynamic chimp optimized efficient Elman neural network (DCO-EENN) is used to return fluency scores based on features retrieved. The results show that AESFSS is highly accurate, with 95% accuracy. The DCO-EENN model attained 0.95 of precision, 0.948 of Recall, and 0.949 of F1-score, which are superior to other spoken English evaluation models.Baseline metrics such as DeepProsody (Yang et al. in Lee, Decentralizing Feature Extraction with Quantum Convolutional Neural Network for Automatic Speech Recognition, 2021), BP Model (Yeung et al. in Alzheimers Res Ther. 10.1186/s13195-021-00848-x, 2021), SVM (Zhang et al. in Circuits Syst Signal Process 41(12):6827–6843, 2022), TDNN (Zhang et al. in Circuits Syst Signal Process 41(12):6827–6843, 2022), and HDE (Zou et al. in System 126, 103497, 2024) are reported from their respective original researches. These models were compared with CMU ARCTIC dataset to better evaluate the performance. This approach realistically captures not only the rhythm but also the natural flow in speech, making the instrument trustworthy for assessing fluency automatically.