DRAMoE: Boosting Adversarial Robustness with Adversarial Training and Adaptive Mixture of Experts
摘要
Deep learning (DL) has been poised to become an integral part of many critical systems. Despite its robustness to natural variations, it is highly susceptible to adversarial examples (AEs) generated by subtle, imperceptible perturbations. As a standard defense form, Adversarial Training (AT) enhances robustness by incorporating AEs during training, yet the distributional gap between clean examples and AEs often compromises standard accuracy, posing a trade-off challenge. By routing examples to matching experts, the Mixture of Experts (MoE) offers a solution. However, current MoE-based AT methods are limited by global pooling-based routers and fail to exploit routing capabilities fully during the AT. This work proposes DRAMoE (Dynamic Router-guided Adversarial Mixture of Experts), a novel MoE architecture integrating AT with a divide-and-conquer strategy. DRAMoE comprises two core components: a Dynamic Channel Attention Router (DCAR) and Router-Guided Adversarial Training (RGAT). DCAR enhances feature representation, while RGAT optimizes AEs generation and designs diverse losses. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny ImageNet demonstrate that DRAMoE improves robust accuracy over the baseline while maintaining competitive standard accuracy.