Spelling correction is a crucial task in Natural Language Processing (NLP), particularly for scarce languages such as Arabic. The aim of this study is to explore the efficacy of Large Language Models (LLMs) in correcting spelling errors in Arabic text. The complexity of Arabic orthography, especially its reliance on contextual dependencies, needs a thorough evaluation of LLMs performance to ensure their reliable and effective deployment in real-world applications. To achieve this, we conduct a comprehensive study of open Arabic LLMs, including AraBERT, CAMeLBERT, ArBERT, MARBERT, mBERT, and XLM-RoBERTa, by testing their ability to generate consistent and relevant predictions for correcting misspelled words. The evaluation is performed using a QALB dataset, ensuring compatibility with the input requirements of aforementioned models. Performance is measured across top-k accuracy and Mean Reciprocal Rank (MRR). Our findings show that AraBERT achieves the highest accuracy at 89,5% exceeding the results achieved by the comparative models in text prediction tasks. The observed performance demonstrates that AraBERT model is well-suited for Arabic spelling correction.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Assessing the Performance of Large Language Models on Arabic Spelling Correction

  • Asmae Regraguy,
  • Saida Laaroussi,
  • Si Lhoussain Aouragh,
  • Said Ouatik El Alaoui

摘要

Spelling correction is a crucial task in Natural Language Processing (NLP), particularly for scarce languages such as Arabic. The aim of this study is to explore the efficacy of Large Language Models (LLMs) in correcting spelling errors in Arabic text. The complexity of Arabic orthography, especially its reliance on contextual dependencies, needs a thorough evaluation of LLMs performance to ensure their reliable and effective deployment in real-world applications. To achieve this, we conduct a comprehensive study of open Arabic LLMs, including AraBERT, CAMeLBERT, ArBERT, MARBERT, mBERT, and XLM-RoBERTa, by testing their ability to generate consistent and relevant predictions for correcting misspelled words. The evaluation is performed using a QALB dataset, ensuring compatibility with the input requirements of aforementioned models. Performance is measured across top-k accuracy and Mean Reciprocal Rank (MRR). Our findings show that AraBERT achieves the highest accuracy at 89,5% exceeding the results achieved by the comparative models in text prediction tasks. The observed performance demonstrates that AraBERT model is well-suited for Arabic spelling correction.