In recent years, more researchers in the field of multi-modal tracking have focused on various algorithms for RGB-T tracking, leveraging the complementary nature of RGB and TIR imaging to achieve good application results. However, their performance tends to degrade in specific scenarios where the target is color-camouflaged and TIR modality is ineffective. Our experiments reveal that ultraviolet (UV) sensors can effectively image certain camouflage materials. Therefore, in extreme scenarios where the target is color-camouflaged and TIR modality is ineffective, UV modality can serve as a supplementary means to RGB and TIR modalities, enhancing tracking performance. In this paper, we propose the first multi-modal object tracking network for visible light, thermal infrared, and ultraviolet, namely VTUTrack, which achieves better tracking performance in complex scenarios. Furthermore, to meet the needs of real-time tracking applications, we introduce an adaptive candidate elimination mechanism based on modality reliability within the ViT (Vision Transformer) backbone network, reducing the computational burden of multi-modal feature extraction and improving tracking inference speed. Extensive experiments further demonstrate the effectiveness of our proposed RGB-T-UV multi-modal object tracking method. The VUOT dataset is available at https://drive.google.com/file/d/1f2FflhUTvfDERJ_j18Plk4XoOur_BLXN/view?usp=drive_link .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Efficient Transformer Network for Visible and Ultraviolet Object Tracking

  • Qinghua Song,
  • Xiaolei Wang

摘要

In recent years, more researchers in the field of multi-modal tracking have focused on various algorithms for RGB-T tracking, leveraging the complementary nature of RGB and TIR imaging to achieve good application results. However, their performance tends to degrade in specific scenarios where the target is color-camouflaged and TIR modality is ineffective. Our experiments reveal that ultraviolet (UV) sensors can effectively image certain camouflage materials. Therefore, in extreme scenarios where the target is color-camouflaged and TIR modality is ineffective, UV modality can serve as a supplementary means to RGB and TIR modalities, enhancing tracking performance. In this paper, we propose the first multi-modal object tracking network for visible light, thermal infrared, and ultraviolet, namely VTUTrack, which achieves better tracking performance in complex scenarios. Furthermore, to meet the needs of real-time tracking applications, we introduce an adaptive candidate elimination mechanism based on modality reliability within the ViT (Vision Transformer) backbone network, reducing the computational burden of multi-modal feature extraction and improving tracking inference speed. Extensive experiments further demonstrate the effectiveness of our proposed RGB-T-UV multi-modal object tracking method. The VUOT dataset is available at https://drive.google.com/file/d/1f2FflhUTvfDERJ_j18Plk4XoOur_BLXN/view?usp=drive_link .