Deep Neural Networks (DNNs) are susceptible to backdoor attacks, where an attacker can insert hidden functionality into DNNs by simply manipulating a small amount of training data, without compromising the victim DNN’s normal functionality. To defend against such attacks, one line of work focuses on detecting suspicious samples before training according to the latent separability assumption that clean and poison samples can be separated in representation space learned by a trained DNN. However, recent strong backdoor attacks can easily break the representation separability, thus existing detection methods become invalid. To this end, we propose to detect poison samples in influence space by tracing data influence on model parameters instead of conventional model outputs. We show that influence separability is significantly stronger than conventional representation separability in terms of four common statistics (e.g., Silhouette Score increases by \(122\%\) on average). With such strong separability in influence space, we can easily obtain stronger backdoor detection and defense by employing existing methods or even simple statistics in influence space. Extensive experiments show that our influence-based methods can significantly outperform conventional representation-based baselines against eight representative backdoor attacks. Particularly, influence space can surprisingly reduce the average attack success rate by 43.4 points ( \(47.2\% \rightarrow 3.8\%\) ) over three benchmark datasets than representation space.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Stronger Separability, Stronger Defense: Influence-Based Backdoor Detection

  • Buhua Liu,
  • Shuo Yang,
  • Zhiqiang Xu,
  • Haoyi Xiong,
  • Yiu-ming Cheung,
  • Zeke Xie

摘要

Deep Neural Networks (DNNs) are susceptible to backdoor attacks, where an attacker can insert hidden functionality into DNNs by simply manipulating a small amount of training data, without compromising the victim DNN’s normal functionality. To defend against such attacks, one line of work focuses on detecting suspicious samples before training according to the latent separability assumption that clean and poison samples can be separated in representation space learned by a trained DNN. However, recent strong backdoor attacks can easily break the representation separability, thus existing detection methods become invalid. To this end, we propose to detect poison samples in influence space by tracing data influence on model parameters instead of conventional model outputs. We show that influence separability is significantly stronger than conventional representation separability in terms of four common statistics (e.g., Silhouette Score increases by \(122\%\) on average). With such strong separability in influence space, we can easily obtain stronger backdoor detection and defense by employing existing methods or even simple statistics in influence space. Extensive experiments show that our influence-based methods can significantly outperform conventional representation-based baselines against eight representative backdoor attacks. Particularly, influence space can surprisingly reduce the average attack success rate by 43.4 points ( \(47.2\% \rightarrow 3.8\%\) ) over three benchmark datasets than representation space.