<p>Omnidirectional images (ODIs) provide a 360° field of view, which necessitates extremely high resolution to capture all the details in the scene. However, their resolution is generally insufficient. Recent research has attempted to address this challenge through image super-resolution (SR) on equirectangular projection (ERP) images. Nonetheless, the translational invariance and self-similarity of the images are compromised by ERP distortion, posing challenges for conventional super-resolution methods to yield optimal outcomes. In this paper, we propose a geometric relationship-guided super-resolution network that incorporates omnidirectional transformer layers (OTLs) and equirectangular convolutions (EquiConvs) to enhance adaptation to distortions in the ERP domain. The OTL module improves feature extraction through the computation of cross-attention between ERP image patches and their corresponding perspective image patches, which are derived from the ERP patches based on their geometric relationship. The EquiConv module demonstrates improved adaptability in addressing distortions by encoding the distortion information into the sampling points of the convolution kernel. The proposed network also employs spatial frequency fusion blocks (SFFB) to enhance its ability to capture global information. Extensive experiments indicate that the proposed method improves objective performance metrics and subjective visual quality in super-resolution reconstruction for ODIs.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Geometric relationship-guided transformer network for omnidirectional image super-resolution

  • Junfeng Cao,
  • Qinghai Ding,
  • Haibo Luo

摘要

Omnidirectional images (ODIs) provide a 360° field of view, which necessitates extremely high resolution to capture all the details in the scene. However, their resolution is generally insufficient. Recent research has attempted to address this challenge through image super-resolution (SR) on equirectangular projection (ERP) images. Nonetheless, the translational invariance and self-similarity of the images are compromised by ERP distortion, posing challenges for conventional super-resolution methods to yield optimal outcomes. In this paper, we propose a geometric relationship-guided super-resolution network that incorporates omnidirectional transformer layers (OTLs) and equirectangular convolutions (EquiConvs) to enhance adaptation to distortions in the ERP domain. The OTL module improves feature extraction through the computation of cross-attention between ERP image patches and their corresponding perspective image patches, which are derived from the ERP patches based on their geometric relationship. The EquiConv module demonstrates improved adaptability in addressing distortions by encoding the distortion information into the sampling points of the convolution kernel. The proposed network also employs spatial frequency fusion blocks (SFFB) to enhance its ability to capture global information. Extensive experiments indicate that the proposed method improves objective performance metrics and subjective visual quality in super-resolution reconstruction for ODIs.