The efficiency and convergence of adversarial training are compromised by the pronounced distributional divergence between clean and adversarial samples, which has been largely attributed to Batch Normalization (BN). Although researchers have attempted to address this mismatch via BN-free or dual-BN frameworks, but these approaches invariably sacrifice natural accuracy for adversarial robustness or vice versa. To overcome these limitations, we introduce Parallel Batch Normalization Adversarial Training (PBNAT), which augments the network with multiple BN branches and a trainable selector that models each input’s feature statistics as a weighted combination of these branches. During training, an alternative BN-scheduling scheme and a novel BN-pruning algorithm work in concert to reduce computational overhead and bolster generalization. During inference, the selector generates a sample-specific weighted combination over all normalization branches, enabling a more flexible and adaptive normalization strategy. This dynamic normalization mechanism enables the model to adapt seamlessly to both clean and adversarial distributions without manual tuning. Empirical results also demonstrate that PBNAT reconciles the accuracy–robustness trade-off, achieving superior natural accuracy and adversarial robustness compared to single-BN, BN-free, and dual-BN baselines.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

PBNAT: Overcoming the Accuracy-Robustness Trade-off via Parallel Batch Normalization

  • Jingning Xu,
  • Chen Chen,
  • Ye Luo,
  • Zhen Gao

摘要

The efficiency and convergence of adversarial training are compromised by the pronounced distributional divergence between clean and adversarial samples, which has been largely attributed to Batch Normalization (BN). Although researchers have attempted to address this mismatch via BN-free or dual-BN frameworks, but these approaches invariably sacrifice natural accuracy for adversarial robustness or vice versa. To overcome these limitations, we introduce Parallel Batch Normalization Adversarial Training (PBNAT), which augments the network with multiple BN branches and a trainable selector that models each input’s feature statistics as a weighted combination of these branches. During training, an alternative BN-scheduling scheme and a novel BN-pruning algorithm work in concert to reduce computational overhead and bolster generalization. During inference, the selector generates a sample-specific weighted combination over all normalization branches, enabling a more flexible and adaptive normalization strategy. This dynamic normalization mechanism enables the model to adapt seamlessly to both clean and adversarial distributions without manual tuning. Empirical results also demonstrate that PBNAT reconciles the accuracy–robustness trade-off, achieving superior natural accuracy and adversarial robustness compared to single-BN, BN-free, and dual-BN baselines.