Well-trained deep neural networks (DNN), including large language models (LLM), are valuable intellectual property assets. To defend against model extraction attacks, one of the major ideas proposed in a large body of previous research is obfuscation: splitting the original DNN and storing the components separately. However, systematically analyzing the methods’ security against various attacks and optimizing the efficiency of defenses are still challenging. In this paper, We propose a taxonomy of model-based extraction attacks, which enables us to identify vulnerabilities of several existing obfuscation methods. We also propose an extremely efficient model obfuscation method called \(\mathsf {O^{2}Splitter}\) using trusted execution environment (TEE). The secrets we store in TEE have \({\mathcal {O}} (1)\) -size, i.e., independent of model size. Although \(\mathsf {O^{2}Splitter}\) relies on a pseudo-random function to provide a quantifiable guarantee for protection and noise compression, it does not need any complicated training or filtering of the weights. Our comprehensive experiments show that \(\mathsf {O^{2}Splitter}\) can mitigate norm-clipping and fine-tuning attacks. Even for small noise ( \(\epsilon = 50 \) ), the accuracy of the obfuscated model is close to random guess, and the tested attacks cannot extract a model with comparable accuracy. In addition, the empirical results also shed light on discovering the relation between DP parameters in obfuscation and the risks of concrete extraction attacks.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Obfuscation for Deep Neural Networks Against Model Extraction: Attack Taxonomy and Defense Optimization

  • Yulian Sun,
  • Vedant Bonde,
  • Li Duan,
  • Yong Li

摘要

Well-trained deep neural networks (DNN), including large language models (LLM), are valuable intellectual property assets. To defend against model extraction attacks, one of the major ideas proposed in a large body of previous research is obfuscation: splitting the original DNN and storing the components separately. However, systematically analyzing the methods’ security against various attacks and optimizing the efficiency of defenses are still challenging. In this paper, We propose a taxonomy of model-based extraction attacks, which enables us to identify vulnerabilities of several existing obfuscation methods. We also propose an extremely efficient model obfuscation method called \(\mathsf {O^{2}Splitter}\) using trusted execution environment (TEE). The secrets we store in TEE have \({\mathcal {O}} (1)\) -size, i.e., independent of model size. Although \(\mathsf {O^{2}Splitter}\) relies on a pseudo-random function to provide a quantifiable guarantee for protection and noise compression, it does not need any complicated training or filtering of the weights. Our comprehensive experiments show that \(\mathsf {O^{2}Splitter}\) can mitigate norm-clipping and fine-tuning attacks. Even for small noise ( \(\epsilon = 50 \) ), the accuracy of the obfuscated model is close to random guess, and the tested attacks cannot extract a model with comparable accuracy. In addition, the empirical results also shed light on discovering the relation between DP parameters in obfuscation and the risks of concrete extraction attacks.