Background <p>Automatic speech recognition (ASR) systems have advanced substantially through transformer-based, self-supervised, multilingual, and weakly supervised learning approaches. However, low-resource languages remain weakly represented in systematic ASR benchmarking, particularly African and Cushitic languages such as Somali. Somali presents additional recognition challenges because of duration-sensitive phonological features, including vowel length and consonant gemination, while standardized Somali ASR evaluation resources remain limited.</p> Methods <p>This study conducted a controlled comparative evaluation of four end-to-end ASR models for Somali: Wav2Vec2.0, XLSR-53, W2V2-BERT, and Whisper-small. The evaluation used a 10.0-h Somali speech test/evaluation dataset containing 6000 utterances from 34 adult native speakers of Standard Somali (Af-Maxaa-Tiri). The dataset was divided into two phonological groups, with 3000 utterances, 17 speakers, and 30 matched reference prompts in each group. Model performance was assessed using word error rate (WER), character error rate (CER), group-wise phonological comparison, paired Wilcoxon signed-rank testing, bootstrap confidence intervals, utterance-level and speaker-level WER distributions, and word-level substitution, deletion, and insertion analysis. All models were evaluated under a controlled no-language-model setting using the same reference transcriptions, preprocessing rules, decoding strategy, and scoring procedure.</p> Results <p>W2V2-BERT achieved the strongest overall performance, with 19.77% WER and 7.14% CER, followed by XLSR-53 with 22.17% WER and 7.98% CER. Whisper-small produced 27.64% WER and 9.86% CER, while Wav2Vec2.0 showed the highest error rates, with 31.04% WER and 10.79% CER. Group-wise results showed slightly higher WER for the higher-complexity phonological group across all models, although the differences were modest. Paired statistical testing showed that W2V2-BERT significantly outperformed XLSR-53, Whisper-small, and Wav2Vec2.0 for both WER and CER. Error-type analysis further showed that W2V2-BERT produced the lowest substitution, deletion, and insertion rates, indicating the most stable recognition profile among the evaluated systems.</p> Conclusions <p>The findings show that model architecture, multilingual transfer, output representation, and evaluation design affect Somali ASR performance. W2V2-BERT and XLSR-53 performed more strongly than Whisper-small and Wav2Vec2.0 under the controlled, no-language-model evaluation setting. The Group A–Group B comparison showed a measurable but limited effect of phonological complexity on recognition accuracy. Therefore, these findings should be interpreted cautiously in relation to the controlled dataset design, matched prompts, speaker distribution, and recording conditions. This study provides a quantitative and phonology-aware benchmark for Somali ASR and identifies the need for larger, more diverse Somali speech datasets, formal phonetic annotation, language-model integration, beam-search evaluation, real-time factor measurement, and deployment-oriented testing.</p>