SpineNet: Learning Scale-Permuted Backbone for Recognition and Localization

CVPR 2020  ·  Xianzhi Du, Tsung-Yi Lin, Pengchong Jin, Golnaz Ghiasi, Mingxing Tan, Yin Cui, Quoc V. Le, Xiaodan Song ·

Convolutional neural networks typically encode an input image into a series of intermediate features with decreasing resolutions. While this structure is suited to classification tasks, it does not perform well for tasks requiring simultaneous recognition and localization (e.g., object detection)... The encoder-decoder architectures are proposed to resolve this by applying a decoder network onto a backbone model designed for classification tasks. In this paper, we argue encoder-decoder architecture is ineffective in generating strong multi-scale features because of the scale-decreased backbone. We propose SpineNet, a backbone with scale-permuted intermediate features and cross-scale connections that is learned on an object detection task by Neural Architecture Search. Using similar building blocks, SpineNet models outperform ResNet-FPN models by ~3% AP at various scales while using 10-20% fewer FLOPs. In particular, SpineNet-190 achieves 52.5% AP with a MaskR-CNN detector and achieves 52.1% AP with a RetinaNet detector on COCO for a single model without test-time augmentation, significantly outperforms prior art of detectors. SpineNet can transfer to classification tasks, achieving 5% top-1 accuracy improvement on a challenging iNaturalist fine-grained dataset. Code is at: https://github.com/tensorflow/tpu/tree/master/models/official/detection. read more

PDF Abstract CVPR 2020 PDF CVPR 2020 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Real-Time Object Detection COCO SpineNet-49 MAP 45.3 # 13
inference time (ms) 34.3 # 10
Real-Time Object Detection COCO SpineNet-49 (RetinaNet, single-scale, with TRT) inference time (ms) 34.3 # 10
Object Detection COCO minival RetinaNet (SpineNet-190, 1536x1536) box AP 52.2 # 11
Instance Segmentation COCO minival RetinaNet (SpineNet-190, 1536x1536) mask AP 46.1 # 6
Object Detection COCO test-dev SpineNet-49 (640, RetinaNet, single-scale) box AP 42.8 # 109
AP50 62.3 # 102
AP75 46.1 # 103
APS 23.7 # 106
APM 45.2 # 106
APL 57.3 # 73
Instance Segmentation COCO test-dev Mask R-CNN (SpineNet-190, 1536x1536) mask AP 46.1 # 7
Object Detection COCO test-dev RetinaNet (SpineNet-49, 896x896) box AP 46.7 # 67
AP50 66.3 # 60
AP75 50.6 # 65
APS 29.1 # 58
APM 50.1 # 56
APL 61.7 # 41
Object Detection COCO test-dev RetinaNet (SpineNet-143, 1280x1280) box AP 50.7 # 38
AP50 70.4 # 28
AP75 54.9 # 36
APS 33.6 # 29
APM 53.9 # 33
APL 62.1 # 37
Object Detection COCO test-dev RetinaNet (SpineNet-190, 1280x1280) box AP 52.1 # 31
AP50 71.8 # 21
AP75 56.5 # 30
APS 35.4 # 19
APM 55 # 27
APL 63.6 # 29
Object Detection COCO test-dev RetinaNet (SpineNet-49S, 640x640) box AP 41.5 # 124
AP50 60.5 # 116
AP75 44.6 # 114
APS 23.3 # 111
APM 45 # 109
APL 58 # 67
Object Detection COCO test-dev RetinaNet (SpineNet-49, 640x640) box AP 44.3 # 91
AP50 63.8 # 87
AP75 47.6 # 87
APS 25.9 # 87
APM 47.7 # 77
APL 61.1 # 46
Object Detection COCO test-dev RetinaNet (SpineNet-96, 1024x1024) box AP 48.6 # 52
AP50 68.4 # 43
AP75 52.5 # 50
APS 32 # 36
APM 52.3 # 42
APL 62 # 40
Image Classification ImageNet SpineNet-143 Top 1 Accuracy 79% # 270
Top 5 Accuracy 94.4% # 128
Number of params 60.5M # 100
Image Classification iNaturalist SpineNet-143 Top 1 Accuracy 63.6% # 4
Top 5 Accuracy 84.8% # 2

Methods