USB: Universal-Scale Object Detection Benchmark

25 Mar 2021 โ€ข Yosuke Shinya

Benchmarks, such as COCO, play a crucial role in object detection. However, existing benchmarks are insufficient in scale variation, and their protocols are inadequate for fair comparison... (read more)

PDF Abstract

Datasets


Introduced in the Paper:

USB

Mentioned in the Paper:

COCO Manga109 Waymo Open Dataset

Results from the Paper


 Ranked #1 on Object Detection on Waymo 2D detection all_ns test (using extra training data)

     Get a GitHub badge
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK USES EXTRA
TRAINING DATA
RESULT BENCHMARK
Real-Time Object Detection COCO UniverseNet-20.08d (Res2Net-101, DCN, single-scale) MAP 51.3 # 4
FPS 11.7 # 20
Real-Time Object Detection COCO UniverseNet-20.08s (ResNet-50-C, DCN, single-scale) MAP 47.4 # 9
FPS 31.6 # 15
Real-Time Object Detection COCO UniverseNet-20.08 (Res2Net-50, DCN, single-scale) MAP 48.8 # 5
FPS 24.9 # 17
Object Detection COCO minival UniverseNet-20.08d (Res2Net-101, DCN, single-scale) box AP 50.9 # 10
AP50 69.5 # 8
AP75 55.4 # 5
APS 33.5 # 4
APM 55.5 # 4
APL 65.8 # 5
Object Detection COCO minival UniverseNet-20.08 (Res2Net-50, DCN, single-scale) box AP 48.5 # 16
AP50 67.0 # 13
AP75 52.6 # 11
APS 30.6 # 6
APM 52.7 # 7
APL 62.7 # 7
Object Detection COCO minival UniverseNet-20.08d (Res2Net-101, DCN, multi-scale) box AP 53.5 # 5
AP50 70.8 # 5
AP75 58.9 # 2
APS 36.9 # 2
APM 57.5 # 2
APL 68.1 # 1
Object Detection COCO test-dev UniverseNet-20.08d (Res2Net-101, DCN, single-scale) box AP 51.3 # 27
AP50 70.0 # 25
AP75 55.8 # 24
APS 31.7 # 32
APM 55.3 # 20
APL 64.9 # 17
Object Detection COCO test-dev UniverseNet-20.08d (Res2Net-101, DCN, multi-scale) box AP 54.1 # 12
AP50 71.6 # 14
AP75 59.9 # 7
APS 35.8 # 9
APM 57.2 # 10
APL 67.4 # 5
Object Detection COCO test-dev UniverseNet-20.08 (Res2Net-50, DCN, single-scale) box AP 48.8 # 42
AP50 67.5 # 41
AP75 53.0 # 37
APS 30.1 # 41
APM 52.3 # 35
APL 61.1 # 39
Object Detection Manga109-s 15test UniverseNet-20.08 COCO-style AP 69.9 # 1
Object Detection Manga109-s 15test UniverseNet COCO-style AP 68.9 # 2
Object Detection Manga109-s 15test Faster R-CNN COCO-style AP 65.8 # 7
Object Detection Manga109-s 15test Cascade R-CNN COCO-style AP 67.6 # 3
Object Detection Manga109-s 15test RetinaNet COCO-style AP 65.3 # 8
Object Detection Manga109-s 15test ATSS COCO-style AP 66.5 # 6
Object Detection Manga109-s 15test ATSEPC COCO-style AP 67.1 # 5
Object Detection Manga109-s 15test GFL COCO-style AP 67.3 # 4
Object Detection USB (Standard USB 1.0 protocol) UniverseNet-20.08 mCAP 52.1 # 1
Object Detection USB (Standard USB 1.0 protocol) Faster R-CNN mCAP 45.9 # 7
Object Detection USB (Standard USB 1.0 protocol) Cascade R-CNN mCAP 48.1 # 3
Object Detection USB (Standard USB 1.0 protocol) RetinaNet mCAP 44.8 # 8
Object Detection USB (Standard USB 1.0 protocol) ATSS mCAP 47.1 # 6
Object Detection USB (Standard USB 1.0 protocol) ATSEPC mCAP 48.1 # 3
Object Detection USB (Standard USB 1.0 protocol) GFL mCAP 47.7 # 5
Object Detection USB (Standard USB 1.0 protocol) UniverseNet mCAP 51.4 # 2
Object Detection Waymo 2D detection all_ns f0val UniverseNet-20.08 COCO-style AP 39.0 # 1
Object Detection Waymo 2D detection all_ns f0val UniverseNet COCO-style AP 38.6 # 2
Object Detection Waymo 2D detection all_ns f0val GFL COCO-style AP 35.7 # 4
Object Detection Waymo 2D detection all_ns f0val ATSEPC COCO-style AP 35.0 # 6
Object Detection Waymo 2D detection all_ns f0val ATSS COCO-style AP 35.4 # 5
Object Detection Waymo 2D detection all_ns f0val RetinaNet COCO-style AP 32.5 # 8
Object Detection Waymo 2D detection all_ns f0val Cascade R-CNN COCO-style AP 36.4 # 3
Object Detection Waymo 2D detection all_ns f0val Faster R-CNN COCO-style AP 34.5 # 7
Object Detection Waymo 2D detection all_ns test UniverseNet AP/L2 67.42 # 1

Methods used in the Paper


METHOD TYPE
๐Ÿค– No Methods Found Help the community by adding them if they're not listed; e.g. Deep Residual Learning for Image Recognition uses ResNet