Unlearning in Large Language Models (LLMs) is critical for ensuring ethical and responsible AI, particularly in mitigating privacy leaks, bias, safety concerns, and adapting to evolving regulations. Existing methods typically depend on retain data or a reference LLM, but they often struggle to balance unlearning effectiveness with preserving model utility. This difficulty arises because fine-tuning with explicit retain data or indirectly relying on a reference LLM’s knowledge—tends to blur the boundary between forgotten and retained information, as similar queries frequently yield overlapping responses. To address this, this chapter proposes eliminating the reliance on retain data or a reference LLM for response calibration. Instead of directly applying gradient ascent on forget data, which commonly leads to instability and poor outcomes, this approach explicitly guides the LLM on what not to answer and, crucially, how to respond based on forget data. This chapter introduces Forget data only Loss AdjustmenT (FLAT), a “flat” loss adjustment method that maximizes the f-divergence between template answers and forget answers solely with respect to forget data. The variational form of this f-divergence provides a principled way to adjust the loss by assigning different importance weights to template responses and to forgetting responses subject to unlearning. Empirical results show that FLAT achieves stronger unlearning performance than existing approaches while minimizing disruption to retained capabilities. This ensures high utility across a wide range of tasks, including copyrighted content unlearning on the Harry Potter dataset and MUSE benchmark, as well as entity unlearning on the TOFU dataset.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Forget Data-Only Optimization for Unlearning

  • Yaxuan Wang

摘要

Unlearning in Large Language Models (LLMs) is critical for ensuring ethical and responsible AI, particularly in mitigating privacy leaks, bias, safety concerns, and adapting to evolving regulations. Existing methods typically depend on retain data or a reference LLM, but they often struggle to balance unlearning effectiveness with preserving model utility. This difficulty arises because fine-tuning with explicit retain data or indirectly relying on a reference LLM’s knowledge—tends to blur the boundary between forgotten and retained information, as similar queries frequently yield overlapping responses. To address this, this chapter proposes eliminating the reliance on retain data or a reference LLM for response calibration. Instead of directly applying gradient ascent on forget data, which commonly leads to instability and poor outcomes, this approach explicitly guides the LLM on what not to answer and, crucially, how to respond based on forget data. This chapter introduces Forget data only Loss AdjustmenT (FLAT), a “flat” loss adjustment method that maximizes the f-divergence between template answers and forget answers solely with respect to forget data. The variational form of this f-divergence provides a principled way to adjust the loss by assigning different importance weights to template responses and to forgetting responses subject to unlearning. Empirical results show that FLAT achieves stronger unlearning performance than existing approaches while minimizing disruption to retained capabilities. This ensures high utility across a wide range of tasks, including copyrighted content unlearning on the Harry Potter dataset and MUSE benchmark, as well as entity unlearning on the TOFU dataset.