Self-Supervised Video Hashing (SSVH) has been widely applied to efficient video retrieval. Existing methods mostly use Siamese-like pairwise training to model inter-sample similarities, yet lack direct optimization for high-quality video content representation, limiting practical performance. To address this issue, we propose a novel SSVH method based on the Mamba-Transformer network (hereafter VHMT), which employs a teacher-student architecture. The teacher model extracts robust low-frequency semantic features using the Discrete Wavelet Transform (DWT) and captures temporal dependencies through a Vision Transformer (ViT)-based Temporal Attention Module (TAM). Concurrently, the introduced Temporal Reconstruction Enhancement (TRE) block further improves fine-grained reconstruction, thereby generating high-quality temporal modeling signals. The student model employs the Mamba architecture to model temporal relationships and achieves knowledge transfer and efficient single-path inference by approximating the teacher model’s feature reconstruction and attention outputs. Furthermore, we introduce a joint loss function to enhance the student model’s approximation of the teacher model’s temporal modeling capability while optimizing the clustering structure of the generated hash codes. Experimental results demonstrate the VHMT outperforms several state-of-the-art methods in mean Average Precision.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Video Hashing via a Mamba-Transformer Network for Retrieval

  • Likai Yang,
  • Nianqiao Li,
  • Xiaoping Liang,
  • Lv Chen,
  • Zhenjun Tang

摘要

Self-Supervised Video Hashing (SSVH) has been widely applied to efficient video retrieval. Existing methods mostly use Siamese-like pairwise training to model inter-sample similarities, yet lack direct optimization for high-quality video content representation, limiting practical performance. To address this issue, we propose a novel SSVH method based on the Mamba-Transformer network (hereafter VHMT), which employs a teacher-student architecture. The teacher model extracts robust low-frequency semantic features using the Discrete Wavelet Transform (DWT) and captures temporal dependencies through a Vision Transformer (ViT)-based Temporal Attention Module (TAM). Concurrently, the introduced Temporal Reconstruction Enhancement (TRE) block further improves fine-grained reconstruction, thereby generating high-quality temporal modeling signals. The student model employs the Mamba architecture to model temporal relationships and achieves knowledge transfer and efficient single-path inference by approximating the teacher model’s feature reconstruction and attention outputs. Furthermore, we introduce a joint loss function to enhance the student model’s approximation of the teacher model’s temporal modeling capability while optimizing the clustering structure of the generated hash codes. Experimental results demonstrate the VHMT outperforms several state-of-the-art methods in mean Average Precision.