Scholarlink

<p>Federated Learning (FL) enables collaborative model training without sharing raw data, but compliance with data privacy regulations such as the “Right to be Forgotten” requires mechanisms to remove specific clients’ contributions from trained models. A key challenge in federated unlearning is selectivity: removing a target client’s influence while preserving performance for other designated clients. This is inherently difficult because deep neural networks learn shared feature representations, and aggressive forgetting operations degrade overall model utility. We propose a Three-Phase Selective Unlearning framework that decouples the conflicting objectives of forgetting and preservation into sequential phases: (1) aggressive forgetting via gradient ascent on the target client’s data, (2) friend restoration through knowledge distillation using the pre-unlearning model as a teacher, and (3) balanced fine-tuning to maintain both objectives simultaneously. By separating these phases, we avoid gradient conflicts that arise from joint optimization. We conduct comprehensive experiments across four datasets—MNIST, CIFAR-10, CIFAR-100 (100 classes), and Tiny-ImageNet (200 classes, <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(64 \times 64\)</EquationSource> </InlineEquation>)—with architectures ranging from small CNNs to ResNet-18 (11.2M parameters) under non-IID settings (Dirichlet <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(\alpha =0.5\)</EquationSource> </InlineEquation>). Our method achieves both target metrics (Forget Accuracy Drop <InlineEquation ID="IEq3"> <EquationSource Format="TEX">\(\ge\)</EquationSource> </InlineEquation>70% and Friend Accuracy <InlineEquation ID="IEq4"> <EquationSource Format="TEX">\(\ge\)</EquationSource> </InlineEquation>70%) on MNIST, and on Tiny-ImageNet achieves 83.6% forget accuracy drop with 76.43% friend accuracy, confirming scalability to realistic settings. Membership inference attack evaluations with a retrained gold-standard comparison further verify effective forgetting. Our results demonstrate that phase decomposition with knowledge distillation is essential for effective selective unlearning in federated settings.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Decoupling forgetting and preservation in federated unlearning via knowledge distillation

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月11日

查看全文
  • Hyun Kwon,
  • Joo Bon Maeng,
  • Dae-Jin Kim

摘要

Federated Learning (FL) enables collaborative model training without sharing raw data, but compliance with data privacy regulations such as the “Right to be Forgotten” requires mechanisms to remove specific clients’ contributions from trained models. A key challenge in federated unlearning is selectivity: removing a target client’s influence while preserving performance for other designated clients. This is inherently difficult because deep neural networks learn shared feature representations, and aggressive forgetting operations degrade overall model utility. We propose a Three-Phase Selective Unlearning framework that decouples the conflicting objectives of forgetting and preservation into sequential phases: (1) aggressive forgetting via gradient ascent on the target client’s data, (2) friend restoration through knowledge distillation using the pre-unlearning model as a teacher, and (3) balanced fine-tuning to maintain both objectives simultaneously. By separating these phases, we avoid gradient conflicts that arise from joint optimization. We conduct comprehensive experiments across four datasets—MNIST, CIFAR-10, CIFAR-100 (100 classes), and Tiny-ImageNet (200 classes, \(64 \times 64\) )—with architectures ranging from small CNNs to ResNet-18 (11.2M parameters) under non-IID settings (Dirichlet \(\alpha =0.5\) ). Our method achieves both target metrics (Forget Accuracy Drop \(\ge\) 70% and Friend Accuracy \(\ge\) 70%) on MNIST, and on Tiny-ImageNet achieves 83.6% forget accuracy drop with 76.43% friend accuracy, confirming scalability to realistic settings. Membership inference attack evaluations with a retrained gold-standard comparison further verify effective forgetting. Our results demonstrate that phase decomposition with knowledge distillation is essential for effective selective unlearning in federated settings.