Synthesize then Compare: Detecting Failures and Anomalies for Semantic Segmentation

ECCV 2020  ·  Yingda Xia, Yi Zhang, Fengze Liu, Wei Shen, Alan Yuille ·

The ability to detect failures and anomalies are fundamental requirements for building reliable systems for computer vision applications, especially safety-critical applications of semantic segmentation, such as autonomous driving and medical image analysis. In this paper, we systematically study failure and anomaly detection for semantic segmentation and propose a unified framework, consisting of two modules, to address these two related problems. The first module is an image synthesis module, which generates a synthesized image from a segmentation layout map, and the second is a comparison module, which computes the difference between the synthesized image and the input image. We validate our framework on three challenging datasets and improve the state-of-the-arts by large margins, \emph{i.e.}, 6% AUPR-Error on Cityscapes, 7% Pearson correlation on pancreatic tumor segmentation in MSD and 20% AUPR on StreetHazards anomaly segmentation.

PDF Abstract ECCV 2020 PDF ECCV 2020 Abstract

Results from the Paper

Ranked #8 on Anomaly Detection on Road Anomaly (using extra training data)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Anomaly Detection Road Anomaly SynthCP AP 24.86 # 8
FPR95 64.69 # 8


No methods listed for this paper. Add relevant methods here