<p>Deep transformer-based neural networks have provided state-of-the-art performances in many computer vision applications, including the task of image super resolution. Specifically, transformer operations efficiently exploit the interactions between various features located at different spatial coordinates and contribute to improving the network performance. Window-based transformers are among the transformer-based neural networks that provide high performances by employing small complexities. Therefore, the use of such operations in the development of deep lightweight high-performance image super resolution networks is of paramount importance. Window-based attention operations mostly focus on processing the local information of the visual signals at a specific hierarchical level. In order to improve the feature generation capability of the window-based transformers for image super resolution, in this paper, we develop new types of attention operations, which utilize the hierarchical, as well as, the spectral information of the visual signals in the feature attention process. The results of various ablation studies show the effectiveness of different ideas employed in the design of the proposed transformer-based neural network for the task of image super resolution. Further, it is demonstrated that the super resolution network using the proposed attention operations is able to provide superior performance than that obtained by state-of-the-art schemes.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

HiSpecmer: a deep efficient image super resolution network using transformers with hierarchical and spectral feature attention

  • Alireza Esmaeilzehi,
  • Hossein Zaredar,
  • Raha Ahmadi,
  • M. Omair Ahmad

摘要

Deep transformer-based neural networks have provided state-of-the-art performances in many computer vision applications, including the task of image super resolution. Specifically, transformer operations efficiently exploit the interactions between various features located at different spatial coordinates and contribute to improving the network performance. Window-based transformers are among the transformer-based neural networks that provide high performances by employing small complexities. Therefore, the use of such operations in the development of deep lightweight high-performance image super resolution networks is of paramount importance. Window-based attention operations mostly focus on processing the local information of the visual signals at a specific hierarchical level. In order to improve the feature generation capability of the window-based transformers for image super resolution, in this paper, we develop new types of attention operations, which utilize the hierarchical, as well as, the spectral information of the visual signals in the feature attention process. The results of various ablation studies show the effectiveness of different ideas employed in the design of the proposed transformer-based neural network for the task of image super resolution. Further, it is demonstrated that the super resolution network using the proposed attention operations is able to provide superior performance than that obtained by state-of-the-art schemes.