The traditional speech transcription pipeline consists of two stages: speech acquisition and the conversion of spoken language into written text using Automatic Speech Recognition (ASR) systems. Despite substantial advances in recent years, ASR technologies continue to exhibit limitations, particularly in handling punctuation, homophones, and linguistic disfluencies. This article proposes an extension of the conventional pipeline by introducing a third stage – post-processing in which generative AI is used to enhance transcription quality. Using a large language model guided by a carefully crafted prompt, raw transcripts are refined for coherence, accuracy, and proper punctuation, while minimizing unintended alterations. An empirical evaluation was conducted on 224 audio samples, including both human and AI-generated voices, with variations in speech rate and three acoustic disturbances: recording distortions, train noise, and crowd noise. Results demonstrate substantial improvements: punctuation restoration, correction of homophone errors in over 43% of cases, and elimination of hesitations or repetitions in 74% of transcripts. Overall, 35% of outputs were subjectively rated as superior, with the most pronounced benefits observed in non-native speakers’ speech.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Towards Reliable Speech-to-Text Systems: Generative AI-Based Context-Aware Post-processing for Punctuation, Homophone Resolution, and Non-native Speech

  • Artur Karczmarczyk,
  • Aleksandra Bączkiewicz,
  • Aleksandra Karczmarczyk,
  • Jakub Więckowski,
  • Grażyna Rosa

摘要

The traditional speech transcription pipeline consists of two stages: speech acquisition and the conversion of spoken language into written text using Automatic Speech Recognition (ASR) systems. Despite substantial advances in recent years, ASR technologies continue to exhibit limitations, particularly in handling punctuation, homophones, and linguistic disfluencies. This article proposes an extension of the conventional pipeline by introducing a third stage – post-processing in which generative AI is used to enhance transcription quality. Using a large language model guided by a carefully crafted prompt, raw transcripts are refined for coherence, accuracy, and proper punctuation, while minimizing unintended alterations. An empirical evaluation was conducted on 224 audio samples, including both human and AI-generated voices, with variations in speech rate and three acoustic disturbances: recording distortions, train noise, and crowd noise. Results demonstrate substantial improvements: punctuation restoration, correction of homophone errors in over 43% of cases, and elimination of hesitations or repetitions in 74% of transcripts. Overall, 35% of outputs were subjectively rated as superior, with the most pronounced benefits observed in non-native speakers’ speech.