Machine learning, as a key supporting technology for AI, has greatly contributed to the rapid development of AI and provided a strong impetus for improving productivity. Meanwhile, machine unlearning has emerged as an important area as data privacy and security concerns become more prominent. It helps to remove certain data knowledge from a trained model without retraining it from scratch. Existing research on machine unlearning methods has primarily focused on improving the efficiency of unlearning algorithms and the effectiveness of data removal. However, it largely ignores whether real-world models can maintain unlearned performance during incremental learning. Motivated by this, we incorporate a knowledge distillation-based feature suppression mechanism to prevent the model from re-learning the removed class representations when the unlearned model undergoes successive incremental learning. By leveraging knowledge distillation, we effectively constrain the feature space, ensuring that the unlearned knowledge does not resurface in subsequent learning stages. Furthermore, we design class-specific unlearning methods to validate the proposed approach and provide a new perspective on class unlearning.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Resisting Catastrophic Recall: Persistent Unlearning via Knowledge Distillation with Feature Suppression

  • Zonghao Ji,
  • Youyang Qu,
  • Longxiang Gao,
  • Taihao Zhang

摘要

Machine learning, as a key supporting technology for AI, has greatly contributed to the rapid development of AI and provided a strong impetus for improving productivity. Meanwhile, machine unlearning has emerged as an important area as data privacy and security concerns become more prominent. It helps to remove certain data knowledge from a trained model without retraining it from scratch. Existing research on machine unlearning methods has primarily focused on improving the efficiency of unlearning algorithms and the effectiveness of data removal. However, it largely ignores whether real-world models can maintain unlearned performance during incremental learning. Motivated by this, we incorporate a knowledge distillation-based feature suppression mechanism to prevent the model from re-learning the removed class representations when the unlearned model undergoes successive incremental learning. By leveraging knowledge distillation, we effectively constrain the feature space, ensuring that the unlearned knowledge does not resurface in subsequent learning stages. Furthermore, we design class-specific unlearning methods to validate the proposed approach and provide a new perspective on class unlearning.