Scholarlink

<p>Convolutional Neural Networks (CNNs) have achieved exceptional success in vision tasks but remain highly susceptible to adversarial perturbations, which limits deployment of these models in safety-sensitive domains. Initial evaluation of Vision Transformers (ViTs), having a token-based architecture and self-attention mechanism, gave improved robustness. However, recent studies reveal that ViTs also suffer from targeted perturbations, notably when attacks get directions of semantically dominant patches. Existing adversarial methods become much more effective with frequency or patch-based cues that do not necessarily correspond to the most decision-relevant features. This work introduces a Singular Value Decomposition (SVD)-guided, attention-aware adversarial framework that identifies and perturbs surrogate models’ decision-critical input image regions within ViT architectures. SVD decomposes the value-projection of the weight matrix to extract high-variance singular directions and fuses them with multi-head attention maps via an exponential moving average (EMA). It produces a soft, saliency-driven guidance mask that prioritizes influential patches while suppressing redundant distortions. The masking integrated with state-of-the-art iterative attacks (MI-FGSM, DI, TI, SI-NI), produces a unified formulation for white-box and transfer adversarial attacks. Experimental results on ImageNet-scale benchmarks demonstrate strong white-box saturation and significantly improved black-box transferability across the architectural diversity of ViT and CNN, achieving up to 80%+ cross-architecture success under modest perturbation budgets. These findings highlight that representation-aware perturbations, which combine attention and spectral priors, can substantially enhance the transferability and efficiency of adversarial attacks on transformer-based vision models.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Cross-architecture robustness evaluation using SVD-guided attention-aware adversarial attacks on vision transformers

  • 期刊论文

  • 发表日期：

    2026年3月10日

查看全文
  • Surya Kant Yadav,
  • Samir Kr. Borgohain

摘要

Convolutional Neural Networks (CNNs) have achieved exceptional success in vision tasks but remain highly susceptible to adversarial perturbations, which limits deployment of these models in safety-sensitive domains. Initial evaluation of Vision Transformers (ViTs), having a token-based architecture and self-attention mechanism, gave improved robustness. However, recent studies reveal that ViTs also suffer from targeted perturbations, notably when attacks get directions of semantically dominant patches. Existing adversarial methods become much more effective with frequency or patch-based cues that do not necessarily correspond to the most decision-relevant features. This work introduces a Singular Value Decomposition (SVD)-guided, attention-aware adversarial framework that identifies and perturbs surrogate models’ decision-critical input image regions within ViT architectures. SVD decomposes the value-projection of the weight matrix to extract high-variance singular directions and fuses them with multi-head attention maps via an exponential moving average (EMA). It produces a soft, saliency-driven guidance mask that prioritizes influential patches while suppressing redundant distortions. The masking integrated with state-of-the-art iterative attacks (MI-FGSM, DI, TI, SI-NI), produces a unified formulation for white-box and transfer adversarial attacks. Experimental results on ImageNet-scale benchmarks demonstrate strong white-box saturation and significantly improved black-box transferability across the architectural diversity of ViT and CNN, achieving up to 80%+ cross-architecture success under modest perturbation budgets. These findings highlight that representation-aware perturbations, which combine attention and spectral priors, can substantially enhance the transferability and efficiency of adversarial attacks on transformer-based vision models.