Federated learning (FL) is a paradigm where multiple clients train a single machine learning model without exposing their training data among them or with a central place. Applying unlearning in this setting is particularly beneficial, in particular when one of the clients decides to leave the federation and wants its data to be removed. While numerous unlearning techniques have been proposed in centralized settings, they cannot be directly applied in the context of distributed settings like federated learning due to the differences in learning protocol and the presence of multiple actors. In this chapter, we first introduce the concept of federated learning and explain why techniques explained in other book chapters are are not suitable for this setting. Then, we overview and contrast existing techniques specially designed for FL. Finally, we dive into one technique that is compatible with complementary multi-party computation commonly applied in FL to efficiently remove a client from a federation. To erase a client, we propose to first perform local unlearning at the client to be erased, and then use the locally unlearned model as the initialization to run very few rounds of federated learning between the server and the remaining clients to obtain the unlearned global model. We empirically compare multiple FL-based unlearning methods by employing multiple performance measures on three datasets, and demonstrate their strengths and weakener.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Unlearning in Federated Learning Settings

  • Anisa Halimi,
  • Swanand Ravinda Kadhe,
  • Ambrish Rawat,
  • Nathalie Baracaldo

摘要

Federated learning (FL) is a paradigm where multiple clients train a single machine learning model without exposing their training data among them or with a central place. Applying unlearning in this setting is particularly beneficial, in particular when one of the clients decides to leave the federation and wants its data to be removed. While numerous unlearning techniques have been proposed in centralized settings, they cannot be directly applied in the context of distributed settings like federated learning due to the differences in learning protocol and the presence of multiple actors. In this chapter, we first introduce the concept of federated learning and explain why techniques explained in other book chapters are are not suitable for this setting. Then, we overview and contrast existing techniques specially designed for FL. Finally, we dive into one technique that is compatible with complementary multi-party computation commonly applied in FL to efficiently remove a client from a federation. To erase a client, we propose to first perform local unlearning at the client to be erased, and then use the locally unlearned model as the initialization to run very few rounds of federated learning between the server and the remaining clients to obtain the unlearned global model. We empirically compare multiple FL-based unlearning methods by employing multiple performance measures on three datasets, and demonstrate their strengths and weakener.