Scholarlink

<p>Accurate traffic object detection constitutes a critical component of intelligent transportation systems (ITS), yet inherent complexities of traffic environments and dynamic illumination conditions present persistent challenges. Visible-infrared (RGB-IR) multi-modal fusion has emerged as a promising paradigm for robust detection by leveraging complementary sensory data. However, existing methods (such as YOLOv8) perform parallel processing on the features of RGB and IR images, and indiscriminately combine modal features through simple fusion methods at the end of the network. These methods often fail to address inherent spatial discrepancies between modalities caused by divergent imaging principles, while naive fusion approaches indiscriminately combine modality features, amplifying noise interference and degrading detection performance. To resolve these limitations, we propose two synergistic innovations: Cross-modality Spatial Feature Modeling (CSFM) and Coordinate Deformable Adjustment and Fusion (CDAF). The CSFM framework establishes a unified subspace for RGB and IR features, jointly suppressing spatial/channel noise while enhancing discriminative feature representations of targets and critical regions across modalities. Complementing this, the CDAF module employs learnable deformable convolutions to dynamically optimize spatial alignment through bidirectional (vertical-horizontal) feature aggregation, simultaneously resolving positional mismatches and capturing long-range contextual dependencies. Comprehensive evaluations on the LLVIP and FLIR benchmarks demonstrate our method’s superiority, achieving state-of-the-art performance metrics. Ablation studies further validate the complementary benefits of CSFM and CDAF in addressing multi-modal discrepancies, demonstrating solid robustness and detection accuracy under diverse environmental conditions.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Deep Multi-modal Feature Fusion for Traffic Object Detection: Cross-modality Spatial Feature Modeling and Coordinate Deformable Adjustment Fusion

  • 期刊论文

  • 发表日期：

    2026年7月7日

查看全文
  • Chenxu Niu,
  • Yuan Feng,
  • Hongliang Li,
  • Pengfei Gao

摘要

Accurate traffic object detection constitutes a critical component of intelligent transportation systems (ITS), yet inherent complexities of traffic environments and dynamic illumination conditions present persistent challenges. Visible-infrared (RGB-IR) multi-modal fusion has emerged as a promising paradigm for robust detection by leveraging complementary sensory data. However, existing methods (such as YOLOv8) perform parallel processing on the features of RGB and IR images, and indiscriminately combine modal features through simple fusion methods at the end of the network. These methods often fail to address inherent spatial discrepancies between modalities caused by divergent imaging principles, while naive fusion approaches indiscriminately combine modality features, amplifying noise interference and degrading detection performance. To resolve these limitations, we propose two synergistic innovations: Cross-modality Spatial Feature Modeling (CSFM) and Coordinate Deformable Adjustment and Fusion (CDAF). The CSFM framework establishes a unified subspace for RGB and IR features, jointly suppressing spatial/channel noise while enhancing discriminative feature representations of targets and critical regions across modalities. Complementing this, the CDAF module employs learnable deformable convolutions to dynamically optimize spatial alignment through bidirectional (vertical-horizontal) feature aggregation, simultaneously resolving positional mismatches and capturing long-range contextual dependencies. Comprehensive evaluations on the LLVIP and FLIR benchmarks demonstrate our method’s superiority, achieving state-of-the-art performance metrics. Ablation studies further validate the complementary benefits of CSFM and CDAF in addressing multi-modal discrepancies, demonstrating solid robustness and detection accuracy under diverse environmental conditions.