<p>Infrared (IR) and visible (VI) image fusion aims to integrate complementary information from heterogeneous modalities to enhance visual perception in a wide range of applications such as remote sensing, military surveillance, and autonomous driving. While transformer-based and diffusion-driven methods have advanced fusion quality, their computational complexity and resource demands limit practical deployment. Furthermore, combining features in a single step may result in the loss of critical information. To address this, we propose PGCFuse, a novel progressive deep learning framework to fuse VI and IR images. The proposed method employs a multilevel progressive feature extraction strategy to preserve multi-scale spatial features, capturing fine-grained local details and global contextual semantics simultaneously. Unlike single-stream hierarchical approaches, our encoder leverages four parallel streams with progressive downsampling, coupled with a multi-scale spatial attention fusion module to adaptively emphasize salient regions across modalities. In particular, we introduce a Gated Convmixer Block that combines depthwise separable convolutions with gating mechanisms to enhance modality-specific feature representation. An Adaptive Global Feature Aggregation block is introduced that models long-range dependencies to refine feature representations. A nested skip-connected decoder reconstructs the fused image while retaining critical structural and thermal information. Comparative evaluations on the benchmark datasets demonstrate that PGCFuse outperforms recent and SOTA DL-based approaches in fusion quality and computational efficiency.<!--Query ID="Q1" Text="Please confirm if the author names are presented accurately and in the correct sequence (given name, middle nameinitial, family name)."--></p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

PGCFuse: a U-Net-based progressive gated convmixer network with global context aggregation for infrared-visible image fusion

  • Hafiz Tayyab Mustafa,
  • Hamza Mustafa,
  • Mujtaba Asad,
  • Syed Farooq Ali,
  • Muhammad Bilal,
  • Muhammad Shehzad Hanif

摘要

Infrared (IR) and visible (VI) image fusion aims to integrate complementary information from heterogeneous modalities to enhance visual perception in a wide range of applications such as remote sensing, military surveillance, and autonomous driving. While transformer-based and diffusion-driven methods have advanced fusion quality, their computational complexity and resource demands limit practical deployment. Furthermore, combining features in a single step may result in the loss of critical information. To address this, we propose PGCFuse, a novel progressive deep learning framework to fuse VI and IR images. The proposed method employs a multilevel progressive feature extraction strategy to preserve multi-scale spatial features, capturing fine-grained local details and global contextual semantics simultaneously. Unlike single-stream hierarchical approaches, our encoder leverages four parallel streams with progressive downsampling, coupled with a multi-scale spatial attention fusion module to adaptively emphasize salient regions across modalities. In particular, we introduce a Gated Convmixer Block that combines depthwise separable convolutions with gating mechanisms to enhance modality-specific feature representation. An Adaptive Global Feature Aggregation block is introduced that models long-range dependencies to refine feature representations. A nested skip-connected decoder reconstructs the fused image while retaining critical structural and thermal information. Comparative evaluations on the benchmark datasets demonstrate that PGCFuse outperforms recent and SOTA DL-based approaches in fusion quality and computational efficiency.