A Two-Branch Fusion Network for Infrared and Visible Image Fusion
摘要
Existing networks for fusing infrared and visible images, whe-ther CNN-based or Transformer-based, struggle to effectively extract and utilize global and local features, leading to poor fusion performance and accuracy. To address this, we propose a two-branch network specifically designed to comprehensively extract and integrate the global and local features of both image types. This network comprises three main components: a global feature extraction module, a detail feature extraction module, and a feature fusion module. The global feature extraction module primarily focuses on capturing global dependencies and high-level semantic information through a self-attention mechanism. This allows it to capture long-range dependencies between different spatial locations in the image, thereby enhancing the understanding of global semantic information. Concurrently, multi-head cross-attention facilitates global interactions between the infrared and visible images, capturing their interdependencies. The detail feature extraction module targets local detail and underlying textures, integrating an Invertible Neural Networks (INN) module with the Convolutional Block Attention Module (CBAM) to ensure lossless information transmission and feature enhancement. Finally, the feature fusion module merges the global and local features based on the activity levels of the row and column vectors, effectively preserving information and minimizing feature conflicts and losses during the fusion process. Experiments on datasets demonstrate that our network outperforms most current methods in terms of metrics and visualization results, and can better preserve critical information and detail such as texture, and contour.