Weakly supervised video anomaly detection uses only video-level labels during the training process to detect frame-level anomalies, featuring low cost and high performance. However, weakly supervised hard labels contain limited information and are subject to certain noise. Additionally, in real-world scenarios, normal and anomalous events are very complex, making it difficult for a single modality of data to encompass sufficient fundamental information. Therefore, our proposes a weakly supervised multimodal video anomaly detection method based on knowledge distillation to address these issues. Specifically, we designed an audio-guided multimodal fusion method that reuses similarity matrices to enhance model performance with minimal additional parameters. Additionally, we proposed a method for generating supervisory signals based on soft labels. We use a general teacher network to generate soft labels and employ a Logits distillation approach to guide our student model in learning more data information. Soft labels can more accurately reflect the true distribution of samples and mimic the decision boundaries of the teacher model, effectively improving the detection performance of the student model. Extensive experiments demonstrate that our method achieves competitive results on two public benchmarks.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Weakly Supervised Multimodal Video Anomaly Detection Based on Knowledge Distillation

  • Lulu Yang,
  • Xiaoyu Wu,
  • Simin Li

摘要

Weakly supervised video anomaly detection uses only video-level labels during the training process to detect frame-level anomalies, featuring low cost and high performance. However, weakly supervised hard labels contain limited information and are subject to certain noise. Additionally, in real-world scenarios, normal and anomalous events are very complex, making it difficult for a single modality of data to encompass sufficient fundamental information. Therefore, our proposes a weakly supervised multimodal video anomaly detection method based on knowledge distillation to address these issues. Specifically, we designed an audio-guided multimodal fusion method that reuses similarity matrices to enhance model performance with minimal additional parameters. Additionally, we proposed a method for generating supervisory signals based on soft labels. We use a general teacher network to generate soft labels and employ a Logits distillation approach to guide our student model in learning more data information. Soft labels can more accurately reflect the true distribution of samples and mimic the decision boundaries of the teacher model, effectively improving the detection performance of the student model. Extensive experiments demonstrate that our method achieves competitive results on two public benchmarks.