<p>Image registration and fusion aim to align multi-modality images, generating a fused image with richer information and higher quality. Existing fusion methods correct spatial misalignment through geometric transformation, semantic guidance, and multi-modality complementarity. However, these methods adopt fixed receptive fields, overlooking local fine-grained features, which leads to structural distortions and edge artifacts. To address these issues, this paper proposes an improved approach for misaligned multi-modality image registration and fusion via feature enhancement, called UnionFusion. Firstly, to achieve the elastic receptive field in registration and fusion tasks, we develop Separable Deformable Convolution (SDConv), which utilizes group-specific learnable sampling offsets to enhance feature representation. Secondly, to address image spatial misalignment, we design Multi-scale Deformable Adaptive Registration module (MDAR) to estimate the deformation field between the source and target images, correcting geometric distortions. Thirdly, to enhance fine-grained features, we design differential fusion mechanism, Shallow Deformable Fusion (SDF) to extract shallow structural features and Deep Progressive Fusion (DPF) to capture deep fine-grained features. Finally, to enhance the complementarity of the joint image registration and fusion, we employ symmetric network and symmetric loss. Extensive experimental analysis demonstrates the effectiveness of the proposed method on four multi-modality image fusion datasets.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Unionfusion: Improving Misaligned Multi-modality Image Registration And Fusion Via Feature Enhancement

  • Yunde Zhang,
  • Jun Kong,
  • Ming Lu,
  • Xuefeng Tao,
  • Min Jiang

摘要

Image registration and fusion aim to align multi-modality images, generating a fused image with richer information and higher quality. Existing fusion methods correct spatial misalignment through geometric transformation, semantic guidance, and multi-modality complementarity. However, these methods adopt fixed receptive fields, overlooking local fine-grained features, which leads to structural distortions and edge artifacts. To address these issues, this paper proposes an improved approach for misaligned multi-modality image registration and fusion via feature enhancement, called UnionFusion. Firstly, to achieve the elastic receptive field in registration and fusion tasks, we develop Separable Deformable Convolution (SDConv), which utilizes group-specific learnable sampling offsets to enhance feature representation. Secondly, to address image spatial misalignment, we design Multi-scale Deformable Adaptive Registration module (MDAR) to estimate the deformation field between the source and target images, correcting geometric distortions. Thirdly, to enhance fine-grained features, we design differential fusion mechanism, Shallow Deformable Fusion (SDF) to extract shallow structural features and Deep Progressive Fusion (DPF) to capture deep fine-grained features. Finally, to enhance the complementarity of the joint image registration and fusion, we employ symmetric network and symmetric loss. Extensive experimental analysis demonstrates the effectiveness of the proposed method on four multi-modality image fusion datasets.