The MVTec 3D-AD Dataset for Unsupervised 3D Anomaly Detection and Localization
We introduce the first comprehensive 3D dataset for the task of unsupervised anomaly detection and localization. It is inspired by real-world visual inspection scenarios in which a model has to detect various types of defects on manufactured products, even if it is trained only on anomaly-free data. There are defects that manifest themselves as anomalies in the geometric structure of an object. These cause significant deviations in a 3D representation of the data. We employed a high-resolution industrial 3D sensor to acquire depth scans of 10 different object categories. For all object categories, we present a training and validation set, each of which solely consists of scans of anomaly-free samples. The corresponding test sets contain samples showing various defects such as scratches, dents, holes, contaminations, or deformations. Precise ground-truth annotations are provided for every anomalous test sample. An initial benchmark of 3D anomaly detection methods on our dataset indicates a considerable room for improvement.
PDF AbstractCode
Datasets
Introduced in the Paper:
MVTEC 3D-AD
Results from the Paper
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
|---|---|---|---|---|---|---|
| 3D Anomaly Detection and Segmentation | MVTEC 3D-AD | Voxel GAN | Segmentation AUPRO | 0.583 | # 6 | |
| Detection AUROC | 0.537 | # 7 | ||||
| RGB+3D Anomaly Detection and Segmentation | MVTEC 3D-AD | Voxel VM | Segmentation AUPRO | 0.471 | # 5 | |
| Detection AUCROC | 0.609 | # 4 | ||||
| RGB+3D Anomaly Detection and Segmentation | MVTEC 3D-AD | Voxel GAN | Segmentation AUPRO | 0.639 | # 3 | |
| Detection AUCROC | 0.517 | # 6 | ||||
| RGB+3D Anomaly Detection and Segmentation | MVTEC 3D-AD | Voxel AE | Segmentation AUPRO | 0.564 | # 4 | |
| Detection AUCROC | 0.538 | # 5 | ||||
| Depth Anomaly Detection and Segmentation | MVTEC 3D-AD | Depth VM | Segmentation AUPRO | 0.374 | # 9 | |
| Detection AUROC | 0.546 | # 9 | ||||
| Depth Anomaly Detection and Segmentation | MVTEC 3D-AD | Depth AE | Segmentation AUPRO | 0.203 | # 10 | |
| Detection AUROC | 0.546 | # 9 | ||||
| Depth Anomaly Detection and Segmentation | MVTEC 3D-AD | Depth GAN | Segmentation AUPRO | 0.143 | # 11 | |
| Detection AUROC | 0.523 | # 11 | ||||
| 3D Anomaly Detection and Segmentation | MVTEC 3D-AD | Voxel VM | Segmentation AUPRO | 0.492 | # 7 | |
| Detection AUROC | 0.571 | # 6 | ||||
| 3D Anomaly Detection and Segmentation | MVTEC 3D-AD | Voxel AE | Segmentation AUPRO | 0.348 | # 8 | |
| Detection AUROC | 0.699 | # 5 |
