<p>Adversarial Training (AT) plays an important role in defending against adversarial attacks, which has become a powerful means to improve the robustness of deep learning models in secure and critical applications. It can be formalized as a min-max optimization problem. At present, the existing adversarial training approaches often use <InlineEquation ID="IEq4"> <EquationSource Format="TEX">\(l_{1}\)</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi>l</mi> <mn>1</mn> </msub> </math></EquationSource> </InlineEquation>, <InlineEquation ID="IEq5"> <EquationSource Format="TEX">\(l_{2}\)</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi>l</mi> <mn>2</mn> </msub> </math></EquationSource> </InlineEquation> and <InlineEquation ID="IEq6"> <EquationSource Format="TEX">\(l_{\infty }\)</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi>l</mi> <mi>∞</mi> </msub> </math></EquationSource> </InlineEquation> norms to constrain the perturbations. However, we find that deep learning models are highly sensitive to <InlineEquation ID="IEq7"> <EquationSource Format="TEX">\(l_{\infty }\)</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi>l</mi> <mi>∞</mi> </msub> </math></EquationSource> </InlineEquation> perturbations. In order to improve the defence of the model against <InlineEquation ID="IEq8"> <EquationSource Format="TEX">\(l_{\infty }\)</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi>l</mi> <mi>∞</mi> </msub> </math></EquationSource> </InlineEquation> perturbations, this paper introduces <InlineEquation ID="IEq9"> <EquationSource Format="TEX">\(l_p\)</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi>l</mi> <mi>p</mi> </msub> </math></EquationSource> </InlineEquation> <InlineEquation ID="IEq10"> <EquationSource Format="TEX">\((0&lt;p&lt;1)\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mo stretchy="false">(</mo> <mn>0</mn> <mo>&lt;</mo> <mi>p</mi> <mo>&lt;</mo> <mn>1</mn> <mo stretchy="false">)</mo> </mrow> </math></EquationSource> </InlineEquation> norm into adversarial training, and proposes <InlineEquation ID="IEq11"> <EquationSource Format="TEX">\(l_p\)</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi>l</mi> <mi>p</mi> </msub> </math></EquationSource> </InlineEquation> norm-based multi steepest descent algorithm (<InlineEquation ID="IEq12"> <EquationSource Format="TEX">\(l_p\)</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi>l</mi> <mi>p</mi> </msub> </math></EquationSource> </InlineEquation>-MSD). Through experiments we find that <InlineEquation ID="IEq13"> <EquationSource Format="TEX">\(l_p\)</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi>l</mi> <mi>p</mi> </msub> </math></EquationSource> </InlineEquation>-MSD algorithm works best when <i>p</i> is equal to 1/2. Furthermore, experimental results show that the robustness of the model is improved by 10.34% and 10.93% compared to traditional multi steepest descent algorithm. Finally, we carry out a perturbation selection experiment to explore the contribution of the <InlineEquation ID="IEq14"> <EquationSource Format="TEX">\(l_{1/2}\)</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi>l</mi> <mrow> <mn>1</mn> <mo stretchy="false">/</mo> <mn>2</mn> </mrow> </msub> </math></EquationSource> </InlineEquation> norm during the training.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

\(l_{1/2}\)-Norm enhanced adversarial training for multiple perturbations

  • Jiaojiao Zhang,
  • Fangfang Xu,
  • Junjie Ma,
  • Yu-Hong Dai

摘要

Adversarial Training (AT) plays an important role in defending against adversarial attacks, which has become a powerful means to improve the robustness of deep learning models in secure and critical applications. It can be formalized as a min-max optimization problem. At present, the existing adversarial training approaches often use \(l_{1}\) l 1 , \(l_{2}\) l 2 and \(l_{\infty }\) l norms to constrain the perturbations. However, we find that deep learning models are highly sensitive to \(l_{\infty }\) l perturbations. In order to improve the defence of the model against \(l_{\infty }\) l perturbations, this paper introduces \(l_p\) l p \((0<p<1)\) ( 0 < p < 1 ) norm into adversarial training, and proposes \(l_p\) l p norm-based multi steepest descent algorithm ( \(l_p\) l p -MSD). Through experiments we find that \(l_p\) l p -MSD algorithm works best when p is equal to 1/2. Furthermore, experimental results show that the robustness of the model is improved by 10.34% and 10.93% compared to traditional multi steepest descent algorithm. Finally, we carry out a perturbation selection experiment to explore the contribution of the \(l_{1/2}\) l 1 / 2 norm during the training.