Assessing the Performance of Large Language Models on Arabic Spelling Correction
摘要
Spelling correction is a crucial task in Natural Language Processing (NLP), particularly for scarce languages such as Arabic. The aim of this study is to explore the efficacy of Large Language Models (LLMs) in correcting spelling errors in Arabic text. The complexity of Arabic orthography, especially its reliance on contextual dependencies, needs a thorough evaluation of LLMs performance to ensure their reliable and effective deployment in real-world applications. To achieve this, we conduct a comprehensive study of open Arabic LLMs, including AraBERT, CAMeLBERT, ArBERT, MARBERT, mBERT, and XLM-RoBERTa, by testing their ability to generate consistent and relevant predictions for correcting misspelled words. The evaluation is performed using a QALB dataset, ensuring compatibility with the input requirements of aforementioned models. Performance is measured across top-k accuracy and Mean Reciprocal Rank (MRR). Our findings show that AraBERT achieves the highest accuracy at 89,5% exceeding the results achieved by the comparative models in text prediction tasks. The observed performance demonstrates that AraBERT model is well-suited for Arabic spelling correction.