Multi-view detection and tracking is essential for pedestrian surveillance and autonomous systems. However, existing methods often overlook the negative impacts of low-quality features from certain viewpoints and feature distortions caused by projection during the feature fusion process. In this paper, we propose a novel multi-view detection and tracking method designed to enhance both multi-view feature extraction and fusion. We propose the View-Aware Attention (VAA) to extract informative and complementary features across diverse camera views, which suppresses the representation of low-quality features in the global feature. Furthermore, we design the Dynamic View-Enhanced Transformer (DVET), which integrates the Dynamic View Augmentation (DVA) module and deformable transformer encoder to dynamically prioritize reliable views and adaptively fuse multi-view features, mitigating the feature distortions caused by projection and insufficient feature fusion. Additionally, we incorporate the Multi-view Gaussian Interpolation (MVGSI) algorithm to further improve tracking continuity. Our method achieves state-of-the-art (SOTA) performance on the Wildtrack (MODA 94.1%, MODP 82.5%, IDF1 97.0%, MOTA 95.2%) and MultiviewX (MODP 93.8%, IDF1 91.3%) datasets, demonstrating superior performance in both detection accuracy and tracking continuity.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

See Through Views: View-Aware Feature Learning for Multi-view Detection and Tracking

  • Zihang Liu,
  • Zirui Li,
  • Chun Liu,
  • Qing Song,
  • Mengjie Hu

摘要

Multi-view detection and tracking is essential for pedestrian surveillance and autonomous systems. However, existing methods often overlook the negative impacts of low-quality features from certain viewpoints and feature distortions caused by projection during the feature fusion process. In this paper, we propose a novel multi-view detection and tracking method designed to enhance both multi-view feature extraction and fusion. We propose the View-Aware Attention (VAA) to extract informative and complementary features across diverse camera views, which suppresses the representation of low-quality features in the global feature. Furthermore, we design the Dynamic View-Enhanced Transformer (DVET), which integrates the Dynamic View Augmentation (DVA) module and deformable transformer encoder to dynamically prioritize reliable views and adaptively fuse multi-view features, mitigating the feature distortions caused by projection and insufficient feature fusion. Additionally, we incorporate the Multi-view Gaussian Interpolation (MVGSI) algorithm to further improve tracking continuity. Our method achieves state-of-the-art (SOTA) performance on the Wildtrack (MODA 94.1%, MODP 82.5%, IDF1 97.0%, MOTA 95.2%) and MultiviewX (MODP 93.8%, IDF1 91.3%) datasets, demonstrating superior performance in both detection accuracy and tracking continuity.