Image Attribution is the task of ascribing importance to regions of the input, for the final decision of a classifier. Many methods for attribution exist, and a recent development has been to use the image at multiple scales to improve the performance of some lightweight attribution methods. These gains have been demonstrated for methods that require a single or multiple but independent forward passes through the network; however, they haven’t been explored in the context of optimization-based attribution (OA) methods. As compared to other techniques, like CAMs or axiomatic methods, OA is attractive as it lends a natural formulation of the task without the need for any heuristic rules. However, the iterative way of solving the optimization problem presents challenges to straightforward utilization of multiple scales. We investigate this scenario and develop a novel 2-step approach that first discovers promising areas across scales and locations from the input and then runs Optimization based Attribution on them. We find that while a naive incorporation of image crops is unsuccessful, this 2-stage pipeline leads to improvements in performance. We provide qualitative and quantitative evidence for this and investigate the reasons for the improvement via multiple ablation experiments.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Using Multiscale Information for Improved Optimization-Based Image Attribution

  • Aniket Singh,
  • Anoop Namboodiri

摘要

Image Attribution is the task of ascribing importance to regions of the input, for the final decision of a classifier. Many methods for attribution exist, and a recent development has been to use the image at multiple scales to improve the performance of some lightweight attribution methods. These gains have been demonstrated for methods that require a single or multiple but independent forward passes through the network; however, they haven’t been explored in the context of optimization-based attribution (OA) methods. As compared to other techniques, like CAMs or axiomatic methods, OA is attractive as it lends a natural formulation of the task without the need for any heuristic rules. However, the iterative way of solving the optimization problem presents challenges to straightforward utilization of multiple scales. We investigate this scenario and develop a novel 2-step approach that first discovers promising areas across scales and locations from the input and then runs Optimization based Attribution on them. We find that while a naive incorporation of image crops is unsuccessful, this 2-stage pipeline leads to improvements in performance. We provide qualitative and quantitative evidence for this and investigate the reasons for the improvement via multiple ablation experiments.