The demand for automatic speech recognition (ASR) systems that support low-resource languages has been growing steadily, especially for languages like Maithili, which are not well-represented in existing ASR models. This research focuses on the fine-tuning of OpenAI’s Whisper model to effectively transcribe Maithili speech into text. The Whisper model, originally trained on a variety of languages, was adapted to Maithili using fine-tuning on a curated dataset of Maithili audio and corresponding transcripts. The study explores the challenges posed by the phonetic and syntactic characteristics of Maithili, which is not in Whisper’s original training data. We applied various data augmentation strategies, including spec-augmentation, pitch-loudness adjustments, and creating custom datasets to expand the training dataset and enhance model robustness. Additionally, the paper investigates the model’s performance under different training conditions and hyperparameter configurations, comparing results with existing baseline systems for Maithili ASR. Preliminary results indicate that the fine-tuned Whisper model significantly improves transcription accuracy. The model achieved a loss ranging from 0.2 to 0.3 and a Word Error Rate (WER) between 45% and 55% across different datasets. These findings highlight the effectiveness of fine-tuning in adapting state-of-the-art ASR models for low-resource languages. This work not only contributes to the development of Maithili speech recognition but also highlights the applicability of Whisper for a broader range of languages in the field of low-resource ASR.