This paper presents methods to improve the accuracy and robustness of multilingual automatic speech recognition (ASR) systems transcribing speech into International Phonetic Alphabet (IPA) sequences. The development of such systems faces considerable challenges, including linguistic diversity, pronunciation variability, and especially the scarcity of high-quality annotated resources for many languages, which hinders model generalization to unseen linguistic domains. We propose a framework that explicitly integrates prior linguistic knowledge into the model training process and leverages auxiliary information via hierarchical multi-task learning (HMTL). The method decomposes phoneme recognition into several levels of abstraction, thus enabling the model to capture both language-independent and language-specific phonetic patterns. Furthermore, we introduce and compare two types of language vector representations, obtained respectively from acoustic signals and from phonetic transcriptions, and evaluate their utility as auxiliary inputs, particularly for low-resource and zero-shot scenarios. Experiments were conducted on multilingual corpora with both high- and low-resource languages, employing a pre-trained Wav2Vec 2.0 architecture as the base model. Baseline models were fine-tuned using Connectionist Temporal Classification (CTC) loss without auxiliary information. Phoneme Error Rate (PER) was used for evaluation in both in-domain and out-of-domain settings. The results demonstrate a relative improvement in recognition accuracy of 7–10% for most scenarios, and an improvement exceeding 20% for out-of-domain languages under reduced training data conditions.