Scholarlink

<p>Unpaired image-to-image translation (I2I) methods achieve strong performance in the visible spectrum, but deteriorate in non-visible modalities such as mid-wave infrared (MWIR) due to severe spectral and radiometric mismatches. In MWIR, object identity is encoded in spatially localized thermal gradients rather than appearance, making conventional perceptual or distribution-matching objectives insufficient for downstream recognition. We address this limitation by proposing a task-aware unpaired translation framework that combines pseudo-pair generation with semantic consistency constraints. Pseudo-paired data are first synthesized using Contrastive Unpaired Translation (CUT) by mapping target-domain images into the source domain. Building on these pairs, we introduce P2PGM, a generative model trained with adversarial and Sobel gradient losses to preserve infrared-specific structural cues, together with a gradient-weighted class activation map (GradCAM)-based semantic loss that aligns task-relevant activations between pseudo-source and generated target images. GradCAM supervision is obtained from a ResNet18 classifier pretrained on real MWIR data, providing domain-specific semantic guidance during translation. On MWIR benchmarks, the proposed framework improves perceptual quality, reducing the Fréchet Inception Distance from 90.10 to 85.21 and increasing SSIM from 0.776 to 0.808. On Cityscapes (Label<InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(\rightarrow \)</EquationSource> <EquationSource Format="MATHML"><math> <mo stretchy="false">→</mo> </math></EquationSource> </InlineEquation>Photo), P2PGM achieves superior semantic consistency, improving mIoU from 24.70 to 26.42 and per-class accuracy from 30.70 to 35.47, while reducing FID from 56.40 to 48.52, as measured using a pretrained DRN segmentation network without fine-tuning. Despite emphasizing infrared characteristics, the method generalizes effectively to RGB domains, demonstrating robust task-aware unpaired translation. The framework trades pixel-level supervision for scalable classifier-driven semantic guidance, enabling practical sim-to-real adaptation when dense labels and paired MWIR data are unavailable.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

CUT-P2PGM: Task-Aware Unpaired Mid-Wave Infrared Image-to-Image Translation via Pseudo-paired Generative Modeling

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年2月13日

查看全文
  • Muhammad Awais Arshad,
  • Haneul Lee,
  • Hosun Lee,
  • Myeongjin Kang,
  • Yeowon Kim,
  • Hyochoong Bang

摘要

Unpaired image-to-image translation (I2I) methods achieve strong performance in the visible spectrum, but deteriorate in non-visible modalities such as mid-wave infrared (MWIR) due to severe spectral and radiometric mismatches. In MWIR, object identity is encoded in spatially localized thermal gradients rather than appearance, making conventional perceptual or distribution-matching objectives insufficient for downstream recognition. We address this limitation by proposing a task-aware unpaired translation framework that combines pseudo-pair generation with semantic consistency constraints. Pseudo-paired data are first synthesized using Contrastive Unpaired Translation (CUT) by mapping target-domain images into the source domain. Building on these pairs, we introduce P2PGM, a generative model trained with adversarial and Sobel gradient losses to preserve infrared-specific structural cues, together with a gradient-weighted class activation map (GradCAM)-based semantic loss that aligns task-relevant activations between pseudo-source and generated target images. GradCAM supervision is obtained from a ResNet18 classifier pretrained on real MWIR data, providing domain-specific semantic guidance during translation. On MWIR benchmarks, the proposed framework improves perceptual quality, reducing the Fréchet Inception Distance from 90.10 to 85.21 and increasing SSIM from 0.776 to 0.808. On Cityscapes (Label \(\rightarrow \) → Photo), P2PGM achieves superior semantic consistency, improving mIoU from 24.70 to 26.42 and per-class accuracy from 30.70 to 35.47, while reducing FID from 56.40 to 48.52, as measured using a pretrained DRN segmentation network without fine-tuning. Despite emphasizing infrared characteristics, the method generalizes effectively to RGB domains, demonstrating robust task-aware unpaired translation. The framework trades pixel-level supervision for scalable classifier-driven semantic guidance, enabling practical sim-to-real adaptation when dense labels and paired MWIR data are unavailable.