Cheating Depth: Enhancing 3D Surface Anomaly Detection via Depth Simulation

2 Nov 2023  ·  Vitjan Zavrtanik, Matej Kristan, Danijel Skočaj ·

RGB-based surface anomaly detection methods have advanced significantly. However, certain surface anomalies remain practically invisible in RGB alone, necessitating the incorporation of 3D information. Existing approaches that employ point-cloud backbones suffer from suboptimal representations and reduced applicability due to slow processing. Re-training RGB backbones, designed for faster dense input processing, on industrial depth datasets is hindered by the limited availability of sufficiently large datasets. We make several contributions to address these challenges. (i) We propose a novel Depth-Aware Discrete Autoencoder (DADA) architecture, that enables learning a general discrete latent space that jointly models RGB and 3D data for 3D surface anomaly detection. (ii) We tackle the lack of diverse industrial depth datasets by introducing a simulation process for learning informative depth features in the depth encoder. (iii) We propose a new surface anomaly detection method 3DSR, which outperforms all existing state-of-the-art on the challenging MVTec3D anomaly detection benchmark, both in terms of accuracy and processing speed. The experimental results validate the effectiveness and efficiency of our approach, highlighting the potential of utilizing depth information for improved surface anomaly detection.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
RGB+3D Anomaly Detection and Segmentation MVTEC 3D-AD 3DSR Segmentation AUPRO 0.972 # 2
Detection AUCROC 0.978 # 1
Segmentation AUCROC 0.995 # 2
Depth Anomaly Detection and Segmentation MVTEC 3D-AD 3DSR (3D) Segmentation AUPRO 0.907 # 5
Detection AUROC 0.922 # 2