Multi-scale hybrid transformer-CNN spatial-spectral prediction architecture for multispectral image compression
摘要
Multispectral images exhibit non-stationarity in the spectral dimension and multi-scale characteristics in the spatial dimension. Currently, many compression methods focus on this aspect. By designing various attention blocks to capture non-stationary and multi-scale features, they have achieved superior performance compared to traditional compression methods in multispectral image compression, with some even outperforming VVC in terms of rate-distortion performance. However, most of these methods are limited within the CNN framework and struggle to effectively capture long-range spectral dependencies. To address the aforementioned issues, we propose a multi-scale hybrid Transformer-CNN spatial-spectral prediction architecture, which effectively overcomes the limitations of CNN in capturing long-range dependencies. This architecture comprises three types of autoencoders, a spectral feature extraction network, and a spatial-spectral prediction network. Specifically, we design a spectral information extractor to effectively extract multi-scale spectral features. In addition, in order to maximize the recovery of rich multi-scale object information in multispectral images, we design a multi-scale prediction network. For balancing the capture of long-range spectral dependencies with the preservation of local texture details, we propose a hybrid Transformer-CNN architecture incorporating spectral multi-head self-attention. This architecture is integrated into the multi-scale prediction network to restore both multi-scale spatial and spectral information effectively. A large number of experiments demonstrate that our method achieves state-of-the-art performance compared to all existing traditional and learnt image compression methods in terms of PSNR, MS-SSIM, Mean Spectral Angle, and subjective evaluation. Code, pre-trained models, training and test datasets are available at https://github.com/xiemulti/MTSPA2025.