Out-of-Distribution Detection without Class Labels

14 Dec 2021  ·  Niv Cohen, Ron Abutbul, Yedid Hoshen ·

Anomaly detection methods identify samples that deviate from the normal behavior of the dataset. It is typically tackled either for training sets containing normal data from multiple labeled classes or a single unlabeled class. Current methods struggle when faced with training data consisting of multiple classes but no labels. In this work, we first discover that classifiers learned by self-supervised image clustering methods provide a strong baseline for anomaly detection on unlabeled multi-class datasets. Perhaps surprisingly, we find that initializing clustering methods with pre-trained features does not improve over their self-supervised counterparts. This is due to the phenomenon of catastrophic forgetting. Instead, we suggest a two stage approach. We first cluster images using self-supervised methods and obtain a cluster label for every image. We use the cluster labels as "pseudo supervision" for out-of-distribution (OOD) methods. Specifically, we finetune pretrained features on the task of classifying images by their cluster labels. We provide extensive analyses of our method and demonstrate the necessity of our two-stage approach. We evaluate it against the state-of-the-art self-supervised and pretrained methods and demonstrate superior performance.

PDF Abstract

Results from the Paper


 Ranked #1 on Anomaly Detection on Unlabeled CIFAR-10 vs CIFAR-100 (using extra training data)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Anomaly Detection Anomaly Detection on Anomaly Detection on Unlabeled ImageNet-30 vs Flowers-102 PsudoLabels CLIP ViT ROC-AUC 98.3 # 1
Network ViT # 1
Anomaly Detection Anomaly Detection on Unlabeled CIFAR-10 vs LSUN (Fix) SCAN Features ROC-AUC 90.2 # 7
Network ResNet-18 # 1
Anomaly Detection Anomaly Detection on Unlabeled CIFAR-10 vs LSUN (Fix) PsudoLabels ResNet-18 ROC-AUC 94.3 # 4
Network ResNet-18 # 1
Anomaly Detection Anomaly Detection on Unlabeled CIFAR-10 vs LSUN (Fix) PsudoLabels ResNet-152 ROC-AUC 95.7 # 3
Network ResNet-152 # 1
Anomaly Detection Anomaly Detection on Unlabeled CIFAR-10 vs LSUN (Fix) PsudoLabels ViT ROC-AUC 99.1 # 1
Network ViT # 1
Anomaly Detection Anomaly Detection on Unlabeled ImageNet-30 vs CUB-200 PsudoLabels CLIP ViT ROC-AUC 99.4 # 1
Network ViT # 1
Anomaly Detection Unlabeled CIFAR-10 vs CIFAR-100 PsudoLabels ResNet-152 AUROC 93.3 # 2
Network ResNet-152 # 1
Anomaly Detection Unlabeled CIFAR-10 vs CIFAR-100 PsudoLabels ResNet-18 AUROC 90.8 # 3
Network ResNet-18 # 1
Anomaly Detection Unlabeled CIFAR-10 vs CIFAR-100 PsudoLabels ViT AUROC 96.7 # 1
Network ViT # 1
Anomaly Detection Unlabeled CIFAR-10 vs CIFAR-100 SCAN Features AUROC 90.2 # 4
Network ResNet-18 # 1

Methods


No methods listed for this paper. Add relevant methods here