DHFCT: dual-branch high-frequency compensate transformer for spectral reconstruction from RGB images
摘要
The reconstruction of hyperspectral images from RGB images is a cost-effective method for retrieving hyperspectral information. However, current spectral reconstruction (SR) methods often overlook the intricacies of feature fusion and the inevitable loss of high-frequency information during feature sampling, directly leading to insufficient reconstruction accuracy and the emergence of noise and artifacts. To mitigate these issues, this study proposes a novel Dual-branch High-Frequency Compensate Transformer (DHFCT) for SR. Specifically, the basic architecture is constituted of several Non-local Dual-Branch Attention Blocks (NDABs), which can achieve better feature fusion performance by employing multi-size windows and adaptive fusion scheme, to fully explore spatial and spectral contextual information. Additionally, the High-Frequency Compensate Module (HFCM) is introduced to compensate for the lost high-frequency information and to enhance the capture ability of spectral differences by leveraging multi-scale features. Extensive experiments on three benchmark datasets demonstrate that DHFCT achieves state-of-the-art performance. Notably, compared with the second-best method on the NTIRE 2022 dataset, our approach achieves improvements of approximately 1.72% in RMSE and 4.07% in SAM.