Automatic speech recognition (ASR) systems often struggle with dysarthric speech, particularly from individuals with neurodegenerative conditions such as Parkinson’s disease. Addressing this challenge, we explore the use of instruction-tuned large language models (LLMs) for ASR error correction. Our approach uses zero-shot prompting with only the 1-best greedy ASR output. Using speech data from the Speech Accessibility Project, we fine-tune a 250M parameter Flan-T5 model to improve transcription accuracy for dysarthric speakers. Evaluation against raw outputs from multiple open-source ASR systems demonstrates notable improvements in correction quality. We further assess larger models, such as the 8B parameter LLaMA 3.1 model, and observe additional gains with minimal hyperparameter tuning. These results highlight the potential of LLM-based correction to enhance ASR accessibility for individuals with speech impairments.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhancing ASR Accuracy for Speakers with Parkinson’s Disease Using Instruction-Tuned LLMs

  • Abner Hernandez,
  • Tomás Arias-Vergara,
  • Andreas Maier,
  • Paula Andrea Pérez-Toro

摘要

Automatic speech recognition (ASR) systems often struggle with dysarthric speech, particularly from individuals with neurodegenerative conditions such as Parkinson’s disease. Addressing this challenge, we explore the use of instruction-tuned large language models (LLMs) for ASR error correction. Our approach uses zero-shot prompting with only the 1-best greedy ASR output. Using speech data from the Speech Accessibility Project, we fine-tune a 250M parameter Flan-T5 model to improve transcription accuracy for dysarthric speakers. Evaluation against raw outputs from multiple open-source ASR systems demonstrates notable improvements in correction quality. We further assess larger models, such as the 8B parameter LLaMA 3.1 model, and observe additional gains with minimal hyperparameter tuning. These results highlight the potential of LLM-based correction to enhance ASR accessibility for individuals with speech impairments.