Scholarlink

<p>Toxic language detection in Arabic remains an underexplored yet critical task, complicated by the language’s dialectal diversity, morphological richness, and the prevalence of overlapping toxicity types. In this paper, AraTox is presented as a large-scale, hand-annotated Arabic dataset designed for multi-label and multi-dialect toxicity classification. The dataset comprises around 36,700 social media comments spanning Gulf, Levantine, Nile Basin, North African, Yemeni, and Modern Standard Arabic (MSA) varieties, each manually labeled by expert annotators across seven categories, including hatred, cussing, racial, appearance-based, and sexual insults. The categories were derived from an iterative refinement of overlapping harassment types. AraTox is benchmarked using a range of deep learning models and classical classifiers, highlighting the superior performance of a stacked meta-learning ensemble over standalone models. The meta-classifier achieves a macro-averaged F1-score of 96% and a subset accuracy of 89.67%, significantly outperforming the Multilingual Arabic BERT, MARBERTv2, and other Bidirectional Encoder Representations from Transformers (BERT)-based baselines; i.e., the meta-classifier achieved +5.31% increase in subset accuracy and +5.23% improvement in precision over the strongest deep learning baseline (MARBERTv2). State-of-the-art large language models (LLMs), including GPT-4 and DeepSeek, are further evaluated in zero- and few-shot settings, demonstrating how they struggle with exact-match accuracy (subset accuracy) and exhibit label-set fragmentation. In addition to quantitative evaluation, a qualitative analysis of representative samples is conducted to identify key challenges in annotating and detecting nuanced forms of toxicity. AraTox sets a new benchmark for Arabic toxicity detection and provides a valuable resource for advancing safer and more inclusive Arabic-language moderation systems. Rather than proposing a novel architecture, this work provides a high-utility benchmark and a multi-label dataset. This includes classical learners, fine-tuned Arabic transformers, a stacked ensemble optimized for multi-label classification, and zero-/few-shot evaluation using LLMs. AraTox is released as an open-source resource to foster collaboration and innovation in the Arabic NLP community.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Aratox: a multi-dialect, multi-label arabic dataset and model benchmark for toxicity detection

  • 期刊论文

  • 发表日期：

    2026年4月17日

查看全文
  • Faisal Alshargi,
  • Ahmed Abulohoom,
  • Sane Yagi,
  • Furat Jabr,
  • Leena Lulu,
  • Ashraf Elnagar

摘要

Toxic language detection in Arabic remains an underexplored yet critical task, complicated by the language’s dialectal diversity, morphological richness, and the prevalence of overlapping toxicity types. In this paper, AraTox is presented as a large-scale, hand-annotated Arabic dataset designed for multi-label and multi-dialect toxicity classification. The dataset comprises around 36,700 social media comments spanning Gulf, Levantine, Nile Basin, North African, Yemeni, and Modern Standard Arabic (MSA) varieties, each manually labeled by expert annotators across seven categories, including hatred, cussing, racial, appearance-based, and sexual insults. The categories were derived from an iterative refinement of overlapping harassment types. AraTox is benchmarked using a range of deep learning models and classical classifiers, highlighting the superior performance of a stacked meta-learning ensemble over standalone models. The meta-classifier achieves a macro-averaged F1-score of 96% and a subset accuracy of 89.67%, significantly outperforming the Multilingual Arabic BERT, MARBERTv2, and other Bidirectional Encoder Representations from Transformers (BERT)-based baselines; i.e., the meta-classifier achieved +5.31% increase in subset accuracy and +5.23% improvement in precision over the strongest deep learning baseline (MARBERTv2). State-of-the-art large language models (LLMs), including GPT-4 and DeepSeek, are further evaluated in zero- and few-shot settings, demonstrating how they struggle with exact-match accuracy (subset accuracy) and exhibit label-set fragmentation. In addition to quantitative evaluation, a qualitative analysis of representative samples is conducted to identify key challenges in annotating and detecting nuanced forms of toxicity. AraTox sets a new benchmark for Arabic toxicity detection and provides a valuable resource for advancing safer and more inclusive Arabic-language moderation systems. Rather than proposing a novel architecture, this work provides a high-utility benchmark and a multi-label dataset. This includes classical learners, fine-tuned Arabic transformers, a stacked ensemble optimized for multi-label classification, and zero-/few-shot evaluation using LLMs. AraTox is released as an open-source resource to foster collaboration and innovation in the Arabic NLP community.