This paper explores the application of machine unlearning techniques to multilingual natural language processing, focusing on Kazakh-language text classification using the XLM-RoBERTa transformer model. The study implements Gradient Ascent Forgetting to selectively degrade the model’s performance on a targeted subset of training data while preserving its general classification capabilities. A dataset of 10,000 annotated texts across categories including bullying, violent extremism, nationalist extremism, racism, and neutral content was used for training and evaluation. Experimental results demonstrate that after applying unlearning, the model’s accuracy on the forgotten class decreased significantly, while overall performance on other classes remained relatively stable, confirming the method’s effectiveness in achieving controlled forgetting. The findings highlight Gradient Ascent Forgetting as a practical and computationally feasible approach for fulfilling data deletion requirements without complete retraining. Limitations and future research directions are discussed, including the trade-off between forgetting precision and model stability, optimization of unlearning parameters, and the extension of this methodology to additional low-resource languages.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Machine Unlearning of Multilingual Transformers for Kazakh Text Classification

  • Milana Bolatbek,
  • Shynar Mussiraliyeva

摘要

This paper explores the application of machine unlearning techniques to multilingual natural language processing, focusing on Kazakh-language text classification using the XLM-RoBERTa transformer model. The study implements Gradient Ascent Forgetting to selectively degrade the model’s performance on a targeted subset of training data while preserving its general classification capabilities. A dataset of 10,000 annotated texts across categories including bullying, violent extremism, nationalist extremism, racism, and neutral content was used for training and evaluation. Experimental results demonstrate that after applying unlearning, the model’s accuracy on the forgotten class decreased significantly, while overall performance on other classes remained relatively stable, confirming the method’s effectiveness in achieving controlled forgetting. The findings highlight Gradient Ascent Forgetting as a practical and computationally feasible approach for fulfilling data deletion requirements without complete retraining. Limitations and future research directions are discussed, including the trade-off between forgetting precision and model stability, optimization of unlearning parameters, and the extension of this methodology to additional low-resource languages.