Back to the Feature: Classical 3D Features are (Almost) All You Need for 3D Anomaly Detection

10 Mar 2022  ·  Eliahu Horwitz, Yedid Hoshen ·

Despite significant advances in image anomaly detection and segmentation, few methods use 3D information. We utilize a recently introduced 3D anomaly detection dataset to evaluate whether or not using 3D information is a lost opportunity. First, we present a surprising finding: standard color-only methods outperform all current methods that are explicitly designed to exploit 3D information. This is counter-intuitive as even a simple inspection of the dataset shows that color-only methods are insufficient for images containing geometric anomalies. This motivates the question: how can anomaly detection methods effectively use 3D information? We investigate a range of shape representations including hand-crafted and deep-learning-based; we demonstrate that rotation invariance plays the leading role in the performance. We uncover a simple 3D-only method that beats all recent approaches while not using deep learning, external pre-training datasets, or color information. As the 3D-only method cannot detect color and texture anomalies, we combine it with color-based features, significantly outperforming previous state-of-the-art. Our method, dubbed BTF (Back to the Feature) achieves pixel-wise ROCAUC: 99.3% and PRO: 96.4% on MVTec 3D-AD.

PDF Abstract

Datasets


Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
3D Anomaly Detection and Segmentation MVTEC 3D-AD Back to the Feature: Classical 3D Features are (Almost) All You Need for 3D Anomaly Detection (FPFH) Segmentation AUPRO 0.924 # 2
Detection AUROC 0.782 # 4
Segmentation AUROC 0.978 # 2
RGB+3D Anomaly Detection and Segmentation MVTEC 3D-AD Back to the Feature: Classical 3D Features are (Almost) All You Need for 3D Anomaly Detection (BTF) Segmentation AUPRO 0.959 # 2
Detection AUCROC 0.865 # 3
Segmentation AUCROC 0.992 # 1
Depth Anomaly Detection and Segmentation MVTEC 3D-AD Back to the Feature: Classical 3D Features are (Almost) All You Need for 3D Anomaly Detection (Depth iNet) Segmentation AUPRO 0.755 # 6
Detection AUROC 0.675 # 6
Segmentation AUROC 0.930 # 5
Depth Anomaly Detection and Segmentation MVTEC 3D-AD Back to the Feature: Classical 3D Features are (Almost) All You Need for 3D Anomaly Detection (NSA) Segmentation AUPRO 0.5572 # 7
Detection AUROC 0.696 # 5
Segmentation AUROC 0.817 # 7
Depth Anomaly Detection and Segmentation MVTEC 3D-AD Back to the Feature: Classical 3D Features are (Almost) All You Need for 3D Anomaly Detection (RaW) Segmentation AUPRO 0.442 # 8
Detection AUROC 0.573 # 7
Segmentation AUROC 0.771 # 8
Depth Anomaly Detection and Segmentation MVTEC 3D-AD Back to the Feature: Classical 3D Features are (Almost) All You Need for 3D Anomaly Detection (HoG) Segmentation AUPRO 0.771 # 5
Detection AUROC 0.559 # 8
Segmentation AUROC 0.930 # 5
Depth Anomaly Detection and Segmentation MVTEC 3D-AD Back to the Feature: Classical 3D Features are (Almost) All You Need for 3D Anomaly Detection (SIFT) Segmentation AUPRO 0.910 # 4
Detection AUROC 0.727 # 4
Segmentation AUROC 0.974 # 3

Methods