WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation

Visual anomaly classification and segmentation are vital for automating industrial quality inspection. The focus of prior research in the field has been on training custom models for each quality inspection task, which requires task-specific images and annotation. In this paper we move away from this regime, addressing zero-shot and few-normal-shot anomaly classification and segmentation. Recently CLIP, a vision-language model, has shown revolutionary generality with competitive zero-/few-shot performance in comparison to full-supervision. But CLIP falls short on anomaly classification and segmentation tasks. Hence, we propose window-based CLIP (WinCLIP) with (1) a compositional ensemble on state words and prompt templates and (2) efficient extraction and aggregation of window/patch/image-level features aligned with text. We also propose its few-normal-shot extension WinCLIP+, which uses complementary information from normal images. In MVTec-AD (and VisA), without further tuning, WinCLIP achieves 91.8%/85.1% (78.1%/79.6%) AUROC in zero-shot anomaly classification and segmentation while WinCLIP+ does 93.1%/95.2% (83.8%/96.4%) in 1-normal-shot, surpassing state-of-the-art by large margins.

PDF Abstract CVPR 2023 PDF CVPR 2023 Abstract

Datasets


Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Anomaly Detection VisA WinCLIP+ (4-shot) Detection AUROC 87.3 # 17
Segmentation AUPRO (until 30% FPR) 87.6 # 10
Anomaly Detection VisA WinCLIP+ (2-shot) Detection AUROC 84.6 # 18
Segmentation AUPRO (until 30% FPR) 86.2 # 11
Anomaly Detection VisA WinCLIP+ (1-shot) Detection AUROC 83.8 # 19
Segmentation AUPRO (until 30% FPR) 85.1 # 13
Anomaly Detection VisA WinCLIP (0-shot) Detection AUROC 78.1 # 23
Segmentation AUPRO (until 30% FPR) 56.8 # 24

Methods