<p>App reviews have become a vital source of feedback for developers seeking to enhance their applications. While considerable research has been conducted on automating user feedback analysis, limited attention has been given to multi-label classification, particularly for Arabic, which is a widely spoken language among app users. The scarcity of multi-label Arabic app review datasets further impedes progress in this area. This paper investigates multi-label classification of Arabic app reviews using pre-trained transformer models. We use a recent dataset labeled with four categories: ratings, improvement requests, bug reports, and others. Additionally, we explore dataset augmentation using ChatGPT-4 to generate synthetic reviews for underrepresented classes. We fine-tune two Arabic transformer models, CamelBert and MarBert, and apply Explainable Artificial Intelligence (XAI) through the Local Interpretable Model-Agnostic Explanation (LIME) technique to analyze model decisions and keyword associations. The results show that ChatGPT-4-based data augmentation enhances model performance. The augmented CamelBert model achieved the highest macro-F1-score of 0.71, while the augmented MarBert model yielded the lowest Hamming loss of 0.275, indicating improved classification accuracy. LIME analysis identified key terms linked to specific labels, highlighted challenges posed by code switching, and revealed labeling inconsistencies within the dataset. Our study underscores the effectiveness of fine-tuned transformer models for multi-label classification of Arabic app reviews, particularly in handling dialectal variation and overlapping categories. Furthermore, integrating data augmentation with interpretability tools such as LIME provides valuable insights into model behavior, especially regarding linguistic complexities like code switching.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multi-label Classification of Arabic App Reviews with Data Augmentation and Explainable AI

  • Aisha Alansari,
  • Dorieh Alomari,
  • Sajjad Mahmood,
  • Irfan Ahmad

摘要

App reviews have become a vital source of feedback for developers seeking to enhance their applications. While considerable research has been conducted on automating user feedback analysis, limited attention has been given to multi-label classification, particularly for Arabic, which is a widely spoken language among app users. The scarcity of multi-label Arabic app review datasets further impedes progress in this area. This paper investigates multi-label classification of Arabic app reviews using pre-trained transformer models. We use a recent dataset labeled with four categories: ratings, improvement requests, bug reports, and others. Additionally, we explore dataset augmentation using ChatGPT-4 to generate synthetic reviews for underrepresented classes. We fine-tune two Arabic transformer models, CamelBert and MarBert, and apply Explainable Artificial Intelligence (XAI) through the Local Interpretable Model-Agnostic Explanation (LIME) technique to analyze model decisions and keyword associations. The results show that ChatGPT-4-based data augmentation enhances model performance. The augmented CamelBert model achieved the highest macro-F1-score of 0.71, while the augmented MarBert model yielded the lowest Hamming loss of 0.275, indicating improved classification accuracy. LIME analysis identified key terms linked to specific labels, highlighted challenges posed by code switching, and revealed labeling inconsistencies within the dataset. Our study underscores the effectiveness of fine-tuned transformer models for multi-label classification of Arabic app reviews, particularly in handling dialectal variation and overlapping categories. Furthermore, integrating data augmentation with interpretability tools such as LIME provides valuable insights into model behavior, especially regarding linguistic complexities like code switching.