Due to the various definitions of anomalies in the real-world, video anomaly detection is a challenging computer vision task. For accurate video anomaly detection, it is important to use multimodal information complementarily to capture the semantic details of the video. In this paper, we propose a multimodal diffusion for video anomaly detection (MDAVD) model using three multimodal features, which enables the complementary use of multimodal information by sharing the distribution of features. The MDVAD consists of two diffusion models, each utilizing text and motion features as conditions, to learn the distribution of normal visual features. The two diffusion models were trained independently and used together during testing to enable complementarity between the two, thereby enabling accurate video anomaly detection. In extensive experiments on two public and two human-related datasets, the proposed model achieved state-of-the-art performances, demonstrating its effectiveness. Code available at https://github.com/kkIIun/MDVAD.git .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

MDVAD: Multimodal Diffusion for Video Anomaly Detection

  • Kijung Lee,
  • Youngwan Jo,
  • Sunghyun Ahn,
  • Sanghyun Park

摘要

Due to the various definitions of anomalies in the real-world, video anomaly detection is a challenging computer vision task. For accurate video anomaly detection, it is important to use multimodal information complementarily to capture the semantic details of the video. In this paper, we propose a multimodal diffusion for video anomaly detection (MDAVD) model using three multimodal features, which enables the complementary use of multimodal information by sharing the distribution of features. The MDVAD consists of two diffusion models, each utilizing text and motion features as conditions, to learn the distribution of normal visual features. The two diffusion models were trained independently and used together during testing to enable complementarity between the two, thereby enabling accurate video anomaly detection. In extensive experiments on two public and two human-related datasets, the proposed model achieved state-of-the-art performances, demonstrating its effectiveness. Code available at https://github.com/kkIIun/MDVAD.git .