Nearest Neighbor Guidance for Out-of-Distribution Detection

26 Sep 2023  ·  Jaewoo Park, Yoon Gyo Jung, Andrew Beng Jin Teoh ·

Detecting out-of-distribution (OOD) samples are crucial for machine learning models deployed in open-world environments. Classifier-based scores are a standard approach for OOD detection due to their fine-grained detection capability. However, these scores often suffer from overconfidence issues, misclassifying OOD samples distant from the in-distribution region. To address this challenge, we propose a method called Nearest Neighbor Guidance (NNGuide) that guides the classifier-based score to respect the boundary geometry of the data manifold. NNGuide reduces the overconfidence of OOD samples while preserving the fine-grained capability of the classifier-based score. We conduct extensive experiments on ImageNet OOD detection benchmarks under diverse settings, including a scenario where the ID data undergoes natural distribution shift. Our results demonstrate that NNGuide provides a significant performance improvement on the base detection scores, achieving state-of-the-art results on both AUROC, FPR95, and AUPR metrics. The code is given at \url{https://github.com/roomo7time/nnguide}.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Out-of-Distribution Detection ImageNet-1k vs Curated OODs (avg.) NNGuide (RegNet) AUROC 95.42 # 3
FPR95 17.97 # 1
Out-of-Distribution Detection ImageNet-1k vs Curated OODs (avg.) NNGuide (ResNet50 w/ ReAct) AUROC 95.45 # 2
FPR95 19.72 # 2
Out-of-Distribution Detection ImageNet-1K vs ImageNet-O NNGuide-ViM (ViT-B/16) AUROC 92.96 # 1
FPR95 33.10 # 1
Out-of-Distribution Detection ImageNet-1k vs iNaturalist NNGuide (ResNet50 w/ ReAct) FPR95 11.12 # 5
AUROC 97.7 # 6
Out-of-Distribution Detection ImageNet-1k vs iNaturalist NNGuide (RegNet) FPR95 1.83 # 1
AUROC 99.57 # 1
Out-of-Distribution Detection ImageNet-1k vs OpenImage-O NNGuide (ResNet50 w/ ReAct) FPR95 35.1 # 3
AUROC 92.49 # 3
Out-of-Distribution Detection ImageNet-1k vs OpenImage-O NNGuide (RegNet) FPR95 10.79 # 1
AUROC 97.73 # 1
Out-of-Distribution Detection ImageNet-1k vs Places NNGuide (RegNet) FPR95 31.47 # 2
AUROC 91.87 # 4
Out-of-Distribution Detection ImageNet-1k vs SUN NNGuide (RegNet) FPR95 21.58 # 2
AUROC 94.43 # 3
Out-of-Distribution Detection ImageNet-1k vs Textures NNGuide (RegNet) FPR95 17.00 # 6
AUROC 95.82 # 7
Out-of-Distribution Detection ImageNet-1k vs Textures NNGuide (ResNet50 w/ ReAct) FPR95 17.27 # 7
AUROC 96.11 # 6

Methods