Machine Unlearning of Multilingual Transformers for Kazakh Text Classification
摘要
This paper explores the application of machine unlearning techniques to multilingual natural language processing, focusing on Kazakh-language text classification using the XLM-RoBERTa transformer model. The study implements Gradient Ascent Forgetting to selectively degrade the model’s performance on a targeted subset of training data while preserving its general classification capabilities. A dataset of 10,000 annotated texts across categories including bullying, violent extremism, nationalist extremism, racism, and neutral content was used for training and evaluation. Experimental results demonstrate that after applying unlearning, the model’s accuracy on the forgotten class decreased significantly, while overall performance on other classes remained relatively stable, confirming the method’s effectiveness in achieving controlled forgetting. The findings highlight Gradient Ascent Forgetting as a practical and computationally feasible approach for fulfilling data deletion requirements without complete retraining. Limitations and future research directions are discussed, including the trade-off between forgetting precision and model stability, optimization of unlearning parameters, and the extension of this methodology to additional low-resource languages.