Leveraging Pre-trained Language Models for Mongolian Intent Classification and Slot Filling
摘要
Intent classification and slot filling enable task-oriented dialogue systems to understand user intent and extract relevant information for task completion. However, the absence of dedicated models and datasets for Mongolian intent classification and slot filling presents a significant challenge in advancing these systems for this low-resource language. To overcome this challenge, we developed the MonNLP2919 dataset, marking the first resource specifically tailored for these tasks. Additionally, we introduced a novel Mongolian intent classification and slot filling model by leveraging a pre-trained language model (CINO) in combination with a Transformer encoder and LSTM layers. Our approach incorporates tailored tokenization strategies for intent classification and slot filling to meet the specific needs of these tasks. Experimental results demonstrate that fine-tuning CINO significantly improves both accuracy and F1-score, highlighting the importance of adapting pre-trained models to the unique linguistic features of Mongolian. These findings emphasize the potential of advanced techniques to improve language processing for low-resource languages. The dataset and code for our experiments are available at https://anonymous.4open.science/r/MN_CINO_intent_slot-79DD .