Speech Recognition Using Spectrogram Images and Formant Data with Neural Networks
摘要
This paper proposes a novel approach to improving speech recognition performance by combining spectrogram images with extracted formant frequency data within a dual input neural network framework. While spectrograms are widely used in speech recognition due to their ability to capture time, frequency, and amplitude information, they may not always effectively highlight the subtle characteristics of formant patterns, especially in low resolution images. To address this, we introduce a hybrid model that integrates spectrogram images with formant parameters derived using the linear predictive coding method. The model utilizes parallel convolutional neural network processing spectrogram images and the other handling time series formant data. Outputs are then concatenated and further processed through fully connected layers. This double channel design enables the network to take advantage of complementary information from both spectral and parametric speech features, leading to improved recognition accuracy. The model is trained and evaluated using the Google Speech Commands dataset, achieving superior results compared to traditional KNN classifiers, formant-only models, and standard spectrogram-based CNNs. The proposed method achieves a 93% recognition accuracy, outperforming conventional approaches and demonstrating the potential of formant-augmented neural networks for more robust speech recognition tasks.