Hyperspectral image (HSI) super-resolution (SR) aims to enhance the spatial resolution of HSIs, conditioning on high-resolution (HR) multispectral images (MSI). Recently, unsupervised learning-based approaches have shown a growing interest in the field of HSI-SR as they do not rely on the supervision of simulated input-output pairs. However, such methods not only require high-performance computational resources but are also prone to overfit. To address this issue, this paper proposes a meta-learning-based HSI-SR method (MetaHSR) that learns the knowledge of a specific HSI via only one model update iteration. During training, knowledge distillation is utilized as a regularization term to avoid overfitting. Furthermore, a Dual-Branch Fusion Transformer (DBFT) is designed, where local, global, and cross-modal dependencies are learned with convolutions, self-attention, and cross-modal attention, respectively. Extensive experiments demonstrate the superiority of the proposed MetaHSR with respect to state-of-the-art methods.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Meta-learning with Dual-Branch Fusion Transformer for Hyperspectral Image Super-Resolution

  • Chanyue Wu,
  • Dong Wang,
  • Changjing Shang,
  • Qiang Shen

摘要

Hyperspectral image (HSI) super-resolution (SR) aims to enhance the spatial resolution of HSIs, conditioning on high-resolution (HR) multispectral images (MSI). Recently, unsupervised learning-based approaches have shown a growing interest in the field of HSI-SR as they do not rely on the supervision of simulated input-output pairs. However, such methods not only require high-performance computational resources but are also prone to overfit. To address this issue, this paper proposes a meta-learning-based HSI-SR method (MetaHSR) that learns the knowledge of a specific HSI via only one model update iteration. During training, knowledge distillation is utilized as a regularization term to avoid overfitting. Furthermore, a Dual-Branch Fusion Transformer (DBFT) is designed, where local, global, and cross-modal dependencies are learned with convolutions, self-attention, and cross-modal attention, respectively. Extensive experiments demonstrate the superiority of the proposed MetaHSR with respect to state-of-the-art methods.