<p>In recent years, performance on existing anomaly detection benchmarks like MVTecAD and VisA has started to saturate in terms of segmentation AU-PRO, with state-of-the-art models often competing in the range of less than one percentage point. This lack of discriminatory power prevents a meaningful comparison of models and thus hinders progress of the field, especially when considering the inherent stochastic nature of machine learning results. We present the MVTecAD2 dataset, a collection of advanced anomaly detection scenarios with more than 8000 high-resolution images from eight object categories. It comprises challenging and highly relevant industrial inspection use cases that have not been considered in previous datasets, including transparent and overlapping objects, dark-field and backlight illumination, objects with high variance in the normal data, and extremely small defects. We provide comprehensive evaluations of state-of-the-art methods and show that their performance remains below 60% average AU-PRO. Additionally, our dataset provides test scenarios with lighting condition changes to assess the robustness of methods under real-world distribution shifts. We host a publicly accessible evaluation server that holds the pixel-precise ground truth of the test set (<a href="https://benchmark.mvtec.com">https://benchmark.mvtec.com</a>). All image data is available at <a href="https://www.mvtec.com/company/research/datasets/mvtec-ad-2">https://www.mvtec.com/company/research/datasets/mvtec-ad-2</a>.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

The MVTec AD 2 Dataset: Advanced Scenarios for Unsupervised Anomaly Detection

  • Lars Heckler-Kram,
  • Jan-Hendrik Neudeck,
  • Ulla Scheler,
  • Rebecca König,
  • Carsten Steger

摘要

In recent years, performance on existing anomaly detection benchmarks like MVTecAD and VisA has started to saturate in terms of segmentation AU-PRO, with state-of-the-art models often competing in the range of less than one percentage point. This lack of discriminatory power prevents a meaningful comparison of models and thus hinders progress of the field, especially when considering the inherent stochastic nature of machine learning results. We present the MVTecAD2 dataset, a collection of advanced anomaly detection scenarios with more than 8000 high-resolution images from eight object categories. It comprises challenging and highly relevant industrial inspection use cases that have not been considered in previous datasets, including transparent and overlapping objects, dark-field and backlight illumination, objects with high variance in the normal data, and extremely small defects. We provide comprehensive evaluations of state-of-the-art methods and show that their performance remains below 60% average AU-PRO. Additionally, our dataset provides test scenarios with lighting condition changes to assess the robustness of methods under real-world distribution shifts. We host a publicly accessible evaluation server that holds the pixel-precise ground truth of the test set (https://benchmark.mvtec.com). All image data is available at https://www.mvtec.com/company/research/datasets/mvtec-ad-2.