Scholarlink

<p>Robust pedestrian detection in real-world scenes remains difficult because of scale variation, body occlusion, and degraded nighttime visibility. RGB-T image pairs offer complementary spectral cues that can mitigate these problems, but most existing methods address each challenge in isolation rather than within a single architecture. We introduce <b>Uni-MPedFormer</b>, a <b>uni</b>fied trans<b>former</b>-based <b>m</b>ultispectral <b>ped</b>estrian detection framework that handles all three modality-shared difficulties within an end-to-end trainable network. The framework consists of three components. First, a Cross-modality Information Fusion Transformer (CIFT) encoder uses the global receptive field of self-attention to capture intra- and inter-modal dependencies between RGB and thermal streams. A Feature Correlation Recalibration (FCR) inner-attention block then enforces neighbourhood-consistent query–key correlations and suppresses spurious responses. Second, a U-shaped Feature Aggregation (UFA) decoder merges multi-scale hierarchical representations through lateral, top-down, and bottom-up pathways, sharpening spatial localisation across pedestrian scales. Third, a lightweight Illumination-aware Weight Generation (IWG) module estimates scene illumination and dynamically re-weights each modality’s contribution to the final detection score. Experiments on five public benchmarks—KAIST, CVC-14, FLIR-ADAS, LLVIP, and UTokyo—and comparisons with recent multimodal detectors show that Uni-MPedFormer achieves competitive or superior accuracy while maintaining a favourable accuracy–speed trade-off (25 FPS on a single 2080Ti). We further report an explicit complexity analysis (parameters, FLOPs, peak memory), a scalability study at higher input resolutions, a sensitivity study on the FCR reduced dimension <i>D</i>, a robustness analysis under simulated RGB-T misalignment, and an explicit module-to-challenge correspondence between CIFT and occlusion, UFA and small-scale pedestrians, and IWG and nighttime detection.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Uni–MPedFormer: towards a unified transformer–based framework for multispectral pedestrian detection

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年6月26日

查看全文
  • Yan Li,
  • Chaoqi Yan,
  • Cunfei Zhao,
  • Jianbo Song

摘要

Robust pedestrian detection in real-world scenes remains difficult because of scale variation, body occlusion, and degraded nighttime visibility. RGB-T image pairs offer complementary spectral cues that can mitigate these problems, but most existing methods address each challenge in isolation rather than within a single architecture. We introduce Uni-MPedFormer, a unified transformer-based multispectral pedestrian detection framework that handles all three modality-shared difficulties within an end-to-end trainable network. The framework consists of three components. First, a Cross-modality Information Fusion Transformer (CIFT) encoder uses the global receptive field of self-attention to capture intra- and inter-modal dependencies between RGB and thermal streams. A Feature Correlation Recalibration (FCR) inner-attention block then enforces neighbourhood-consistent query–key correlations and suppresses spurious responses. Second, a U-shaped Feature Aggregation (UFA) decoder merges multi-scale hierarchical representations through lateral, top-down, and bottom-up pathways, sharpening spatial localisation across pedestrian scales. Third, a lightweight Illumination-aware Weight Generation (IWG) module estimates scene illumination and dynamically re-weights each modality’s contribution to the final detection score. Experiments on five public benchmarks—KAIST, CVC-14, FLIR-ADAS, LLVIP, and UTokyo—and comparisons with recent multimodal detectors show that Uni-MPedFormer achieves competitive or superior accuracy while maintaining a favourable accuracy–speed trade-off (25 FPS on a single 2080Ti). We further report an explicit complexity analysis (parameters, FLOPs, peak memory), a scalability study at higher input resolutions, a sensitivity study on the FCR reduced dimension D, a robustness analysis under simulated RGB-T misalignment, and an explicit module-to-challenge correspondence between CIFT and occlusion, UFA and small-scale pedestrians, and IWG and nighttime detection.