Scholarlink

<p>Current RGB-thermal video object detection (RGBT VOD) methods predominantly depend on aligned multimodal image pairs, that not only bring labor-intensive manual annotation but also introduce potential dataset biases. Although recent studies have explored alignment-free RGBT VOD, these works still depend on manual rule-based guidance, degrading the generalization capability and bringing barriers to real-world applications. In this paper, we propose a multi-graph cross diffusion attention network for alignment-free RGBT VOD. The core contributions are two aspects. First, we introduce a new alignment-sensitive transformation to automatically learn the scaling and translation parameters, enabling <i>preliminary</i> spatial alignment without manual intervention. Then, we develop a novel multi-graph cross diffusion attention (MCDA) module to effectively aggregate the spatio-temporal contextual information of RGB and thermal images. The core of MCDA lies in constructing two modality-independent graphs to capture the dependencies of RGBT features. It facilitates the modeling of cross-modal feature relationships and fully leverages the complementary information between modalities in graph space, which can well alleviate the issue of the modality gap in the fusion process. Extensive experiments demonstrate that our method achieves state-of-the-art performance on the unaligned UVT-VOD2024 benchmark. Moreover, it maintains competitive performance on the aligned VT-VOD50 dataset. We will release our code at <a href="https://github.com/tzz-ahu/MCDANet">https://github.com/tzz-ahu/MCDANet</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Multi-graph Cross Diffusion Attention Network for Alignment-free RGBT Video Object Detection

  • 期刊论文

  • 发表日期：

    2026年7月30日

查看全文
  • Qishun Wang,
  • Zhengzheng Tu,
  • Bo Jiang,
  • Chenglong Li,
  • Pingping Zhang,
  • Jin Tang

摘要

Current RGB-thermal video object detection (RGBT VOD) methods predominantly depend on aligned multimodal image pairs, that not only bring labor-intensive manual annotation but also introduce potential dataset biases. Although recent studies have explored alignment-free RGBT VOD, these works still depend on manual rule-based guidance, degrading the generalization capability and bringing barriers to real-world applications. In this paper, we propose a multi-graph cross diffusion attention network for alignment-free RGBT VOD. The core contributions are two aspects. First, we introduce a new alignment-sensitive transformation to automatically learn the scaling and translation parameters, enabling preliminary spatial alignment without manual intervention. Then, we develop a novel multi-graph cross diffusion attention (MCDA) module to effectively aggregate the spatio-temporal contextual information of RGB and thermal images. The core of MCDA lies in constructing two modality-independent graphs to capture the dependencies of RGBT features. It facilitates the modeling of cross-modal feature relationships and fully leverages the complementary information between modalities in graph space, which can well alleviate the issue of the modality gap in the fusion process. Extensive experiments demonstrate that our method achieves state-of-the-art performance on the unaligned UVT-VOD2024 benchmark. Moreover, it maintains competitive performance on the aligned VT-VOD50 dataset. We will release our code at https://github.com/tzz-ahu/MCDANet.