MVTec AD 2数据集:无监督异常检测的高级场景
The MVTec AD 2 Dataset: Advanced Scenarios for Unsupervised Anomaly Detection
摘要 Abstract
近年来,现有异常检测基准(如MVTec AD和VisA)在分割AU-PRO指标上的性能已开始饱和,最先进的模型往往仅相差不到一个百分点。这种缺乏区分能力阻碍了对模型的有意义比较,从而阻碍了该领域的进步,尤其是考虑到机器学习结果固有的随机性时。我们推出了MVTec AD 2,这是一组包含八种异常检测场景的数据集,拥有超过8000张高分辨率图像。它包含了之前数据集中未考虑的具有挑战性和高度相关的工业检测用例,包括透明和重叠物体、暗场和背光照明、正常数据中存在高变异性以及极小缺陷的情况。我们对最先进的方法进行了全面评估,并表明其性能仍低于60%平均AU-PRO。此外,我们的数据集提供了光照条件变化的测试场景,用于评估方法在实际分布偏移下的鲁棒性。我们提供了一个公开可访问的评估服务器,其中包含测试集的像素精确地面真实标签(https://benchmark.mvtec.com/)。所有图像数据均可在https://www.mvtec.com/company/research/datasets/mvtec-ad-2获取。
In recent years, performance on existing anomaly detection benchmarks like MVTec AD and VisA has started to saturate in terms of segmentation AU-PRO, with state-of-the-art models often competing in the range of less than one percentage point. This lack of discriminatory power prevents a meaningful comparison of models and thus hinders progress of the field, especially when considering the inherent stochastic nature of machine learning results. We present MVTec AD 2, a collection of eight anomaly detection scenarios with more than 8000 high-resolution images. It comprises challenging and highly relevant industrial inspection use cases that have not been considered in previous datasets, including transparent and overlapping objects, dark-field and back light illumination, objects with high variance in the normal data, and extremely small defects. We provide comprehensive evaluations of state-of-the-art methods and show that their performance remains below 60% average AU-PRO. Additionally, our dataset provides test scenarios with lighting condition changes to assess the robustness of methods under real-world distribution shifts. We host a publicly accessible evaluation server that holds the pixel-precise ground truth of the test set (https://benchmark.mvtec.com/). All image data is available at https://www.mvtec.com/company/research/datasets/mvtec-ad-2.