Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming

ICLR 2020 Claudio MichaelisBenjamin MitzkusRobert GeirhosEvgenia RusakOliver BringmannAlexander S. EckerMatthias BethgeWieland Brendel

The ability to detect objects regardless of image distortions or weather conditions is crucial for real-world applications of deep learning like autonomous driving. We here provide an easy-to-use benchmark to assess how object detection models perform when image quality degrades... (read more)

PDF Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT LEADERBOARD
Robust Object Detection Cityscapes test Faster R-CNN mPC [AP] 12.2 # 2
rPC [%] 33.4 # 2
Robust Object Detection Cityscapes test Faster R-CNN with Stylized Training Data mPC [AP] 17.2 # 1
rPC [%] 47.4 # 1
Robust Object Detection COCO Faster R-CNN with Stylized Training Data mPC [AP] 20.4 # 1
rPC [%] 58.9 # 1
Robust Object Detection COCO Faster R-CNN mPC [AP] 18.2 # 2
rPC [%] 50.2 # 2
Robust Object Detection PASCAL VOC 2007 Faster R-CNN mPC [AP50] 48.6 # 2
rPC [%] 60.4 # 2
Robust Object Detection PASCAL VOC 2007 Faster R-CNN with Stylized Training Data mPC [AP50] 56.2 # 1
rPC [%] 69.9 # 1