End-to-end object detection models such as DETR and its variants have received widespread attention due to their unique architectural design and impressive detection performance. However, adversarial attacks targeting these models remain relatively underexplored. Existing approaches have primarily focused on traditional two-stage or anchor-based detectors, making them difficult to directly apply to the DETR architecture. To obtain a deeper insight into the behavioral characteristics of DETR-like models under adversarial perturbations, this paper introduces a structured attack strategy based on directional gradient projection, focusing on four categories of attack objectives: object vanishing, misclassification, object fabrication, and random output. This method is implemented through guided projection of gradients toward targeted directions, achieved by intervening in the bipartite matching mechanism, perturbing the decision boundaries of the classification head, reinforcing the self-enhancement mechanism, and disrupting the collaboration of feature streams, thereby enabling the model to generate controllable adversarial misdetections at the output stage. Comprehensive evaluations conducted on the MS COCO dataset reveal that the proposed attack methods cause an average AP reduction of 0.378 across different DETR variants, with the decline reaching 0.408 under the object-vanishing objective. These results significantly degrade detection performance and expose the structural vulnerability of DETR-based models when subjected to adversarial perturbations.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Directional Gradient Attacks for Inducing Controllable Detection Errors in DETR-Based Models

  • Shujian Liao,
  • Shengyu Xiong,
  • Fudie Ai,
  • Wanli Dong,
  • Yong Peng,
  • Anjie Peng,
  • Hui Zeng

摘要

End-to-end object detection models such as DETR and its variants have received widespread attention due to their unique architectural design and impressive detection performance. However, adversarial attacks targeting these models remain relatively underexplored. Existing approaches have primarily focused on traditional two-stage or anchor-based detectors, making them difficult to directly apply to the DETR architecture. To obtain a deeper insight into the behavioral characteristics of DETR-like models under adversarial perturbations, this paper introduces a structured attack strategy based on directional gradient projection, focusing on four categories of attack objectives: object vanishing, misclassification, object fabrication, and random output. This method is implemented through guided projection of gradients toward targeted directions, achieved by intervening in the bipartite matching mechanism, perturbing the decision boundaries of the classification head, reinforcing the self-enhancement mechanism, and disrupting the collaboration of feature streams, thereby enabling the model to generate controllable adversarial misdetections at the output stage. Comprehensive evaluations conducted on the MS COCO dataset reveal that the proposed attack methods cause an average AP reduction of 0.378 across different DETR variants, with the decline reaching 0.408 under the object-vanishing objective. These results significantly degrade detection performance and expose the structural vulnerability of DETR-based models when subjected to adversarial perturbations.