Scholarlink

<p>Multimodal multi-object tracking now constitutes a foundational capability of autonomous driving platforms, allowing intelligent vehicles to perceive and continuously track dynamic agents through the joint exploitation of visual, auditory, and linguistic signals. Nevertheless, the problem remains hindered by two enduring challenges. One lies in the limited effectiveness of multimodal fusion—most notably the neglect of frequency-domain cues when representation learning is confined to spatial processing alone. The other concerns the instability of temporal semantic alignment, a weakness that often manifests as cumulative tracking drift over consecutive frames. To address these issues, prior studies have explored spatial–frequency fusion mechanisms alongside contrastive learning strategies designed to align heterogeneous modality embeddings. Although such efforts have yielded measurable improvements, their capacity to accommodate complex cross-modal interactions and volatile long-term temporal dependencies remains constrained. Motivated by these limitations, we propose <b>SynTrackThinking</b>, an integrated multimodal tracking framework that tightly couples frequency-aware representation learning with contrastive temporal reasoning. The framework is built upon two core components: an <b>explainable multimodal-domain cross-attention fusion module (EMCFM)</b>, which enhances representational expressiveness via coordinated attention across spatial and frequency domains, and a <b>multimodal contrastive tracking learning (MCTL)</b> strategy, explicitly designed to preserve semantic and temporal coherence among modalities over time. Extensive experiments conducted on newly developed multimodal and text-guided benchmarks demonstrate that SynTrackThinking achieves consistent gains in tracking precision, robustness, and generalization across a wide range of operating conditions, underscoring its suitability as a scalable and dependable solution for real-world autonomous tracking scenarios. The source code and datasets are available at <a href="https://zenodo.org/records/18824268">https://zenodo.org/records/18824268</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

SynTrackThinking improves multimodal multi-object tracking for autonomous driving through frequency-aware fusion and temporal contrastive learning

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年3月17日

查看全文
  • Zhihao Su,
  • Wei Chang

摘要

Multimodal multi-object tracking now constitutes a foundational capability of autonomous driving platforms, allowing intelligent vehicles to perceive and continuously track dynamic agents through the joint exploitation of visual, auditory, and linguistic signals. Nevertheless, the problem remains hindered by two enduring challenges. One lies in the limited effectiveness of multimodal fusion—most notably the neglect of frequency-domain cues when representation learning is confined to spatial processing alone. The other concerns the instability of temporal semantic alignment, a weakness that often manifests as cumulative tracking drift over consecutive frames. To address these issues, prior studies have explored spatial–frequency fusion mechanisms alongside contrastive learning strategies designed to align heterogeneous modality embeddings. Although such efforts have yielded measurable improvements, their capacity to accommodate complex cross-modal interactions and volatile long-term temporal dependencies remains constrained. Motivated by these limitations, we propose SynTrackThinking, an integrated multimodal tracking framework that tightly couples frequency-aware representation learning with contrastive temporal reasoning. The framework is built upon two core components: an explainable multimodal-domain cross-attention fusion module (EMCFM), which enhances representational expressiveness via coordinated attention across spatial and frequency domains, and a multimodal contrastive tracking learning (MCTL) strategy, explicitly designed to preserve semantic and temporal coherence among modalities over time. Extensive experiments conducted on newly developed multimodal and text-guided benchmarks demonstrate that SynTrackThinking achieves consistent gains in tracking precision, robustness, and generalization across a wide range of operating conditions, underscoring its suitability as a scalable and dependable solution for real-world autonomous tracking scenarios. The source code and datasets are available at https://zenodo.org/records/18824268.