Accurate and timely pronunciation feedback remains a significant barrier for second-language learners, particularly adults aged 18–35 with intermediate to upper-intermediate English proficiency (CEFR B1–C1). This paper introduces a hybrid, deep learning-based framework that addresses this challenge by integrating Whisper Automatic Speech Recognition (ASR), a phoneme-aware Generative Adversarial Net- work (GAN), and GPT-4o for contextual feedback generation. The system captures real-time speech, performs IPA-based phoneme alignment via Epitran, and applies Dynamic Time Warping (DTW) to assess articulation accuracy. A GAN model trained on data from the L2-ARCTIC and CMU Wilderness datasets simulates native phoneme embeddings, identifying subtle pronunciation deviations. GPT-4o generates linguistically-informed, personalized feedback in both textual and synthesized audio formats using NeuralTTS. The pipeline is deployed as stateless microservices on AWS Lambda to ensure elastic scalability and low-latency processing. Experimental results demonstrate significant gains in transcription accuracy (from 98.82% to 99.83%) and user-rated feedback clarity (from 55.88% to 88.98%). While current support is limited to English and German, this approach lays the groundwork for scalable, real-time pronunciation training across diverse languages and contexts.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Pronunciation Generation and Correction Using GAN and GPT-4o

  • Gaurav Indra,
  • Khushi Sharma,
  • Ayushi Gupta,
  • Kinjal Singh

摘要

Accurate and timely pronunciation feedback remains a significant barrier for second-language learners, particularly adults aged 18–35 with intermediate to upper-intermediate English proficiency (CEFR B1–C1). This paper introduces a hybrid, deep learning-based framework that addresses this challenge by integrating Whisper Automatic Speech Recognition (ASR), a phoneme-aware Generative Adversarial Net- work (GAN), and GPT-4o for contextual feedback generation. The system captures real-time speech, performs IPA-based phoneme alignment via Epitran, and applies Dynamic Time Warping (DTW) to assess articulation accuracy. A GAN model trained on data from the L2-ARCTIC and CMU Wilderness datasets simulates native phoneme embeddings, identifying subtle pronunciation deviations. GPT-4o generates linguistically-informed, personalized feedback in both textual and synthesized audio formats using NeuralTTS. The pipeline is deployed as stateless microservices on AWS Lambda to ensure elastic scalability and low-latency processing. Experimental results demonstrate significant gains in transcription accuracy (from 98.82% to 99.83%) and user-rated feedback clarity (from 55.88% to 88.98%). While current support is limited to English and German, this approach lays the groundwork for scalable, real-time pronunciation training across diverse languages and contexts.