Complementary Pseudo Multimodal Feature for Point Cloud Anomaly Detection

23 Mar 2023  ·  Yunkang Cao, Xiaohao Xu, Weiming Shen ·

Point cloud (PCD) anomaly detection steadily emerges as a promising research area. This study aims to improve PCD anomaly detection performance by combining handcrafted PCD descriptions with powerful pre-trained 2D neural networks. To this end, this study proposes Complementary Pseudo Multimodal Feature (CPMF) that incorporates local geometrical information in 3D modality using handcrafted PCD descriptors and global semantic information in the generated pseudo 2D modality using pre-trained 2D neural networks. For global semantics extraction, CPMF projects the origin PCD into a pseudo 2D modality containing multi-view images. These images are delivered to pre-trained 2D neural networks for informative 2D modality feature extraction. The 3D and 2D modality features are aggregated to obtain the CPMF for PCD anomaly detection. Extensive experiments demonstrate the complementary capacity between 2D and 3D modality features and the effectiveness of CPMF, with 95.15% image-level AU-ROC and 92.93% pixel-level PRO on the MVTec3D benchmark. Code is available on https://github.com/caoyunkang/CPMF.

PDF Abstract

Datasets


Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Benchmark
Depth Anomaly Detection and Segmentation MVTEC 3D-AD CPMF (2D+3D) Segmentation AUPRO 0.9293 # 1
Detection AUROC 0.9515 # 1
Segmentation AUROC 0.9781 # 1
3D Anomaly Detection and Segmentation MVTEC 3D-AD CPMF (2D+3D) Segmentation AUPRO 0.9293 # 2
Detection AUROC 0.9515 # 1
Segmentation AUROC 0.9781 # 1
3D Anomaly Detection and Segmentation MVTEC 3D-AD CPMF (3D) Segmentation AUPRO 0.9230 # 4
Detection AUROC 0.8304 # 4
Segmentation AUROC 0.9780 # 2
3D Anomaly Detection and Segmentation MVTEC 3D-AD CPMF (2D) Segmentation AUPRO 0.9145 # 5
Detection AUROC 0.8918 # 3
Segmentation AUROC 0.9730 # 5
Depth Anomaly Detection and Segmentation MVTEC 3D-AD CPMF (2D) Segmentation AUPRO 0.9145 # 3
Detection AUROC 0.8918 # 3
Segmentation AUROC 0.9730 # 4
Depth Anomaly Detection and Segmentation MVTEC 3D-AD CPMF (3D) Segmentation AUPRO 0.9230 # 2
Detection AUROC 0.8304 # 4
Segmentation AUROC 0.9780 # 2

Methods


No methods listed for this paper. Add relevant methods here