Feature Pyramid Networks for Object Detection

Feature pyramids are a basic component in recognition systems for detecting objects at different scales. But recent deep learning object detectors have avoided pyramid representations, in part because they are compute and memory intensive. In this paper, we exploit the inherent multi-scale, pyramidal hierarchy of deep convolutional networks to construct feature pyramids with marginal extra cost. A top-down architecture with lateral connections is developed for building high-level semantic feature maps at all scales. This architecture, called a Feature Pyramid Network (FPN), shows significant improvement as a generic feature extractor in several applications. Using FPN in a basic Faster R-CNN system, our method achieves state-of-the-art single-model results on the COCO detection benchmark without bells and whistles, surpassing all existing single-model entries including those from the COCO 2016 challenge winners. In addition, our method can run at 5 FPS on a GPU and thus is a practical and accurate solution to multi-scale object detection. Code will be made publicly available.

PDF Abstract CVPR 2017 PDF CVPR 2017 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Object Detection COCO minival FPN+ box AP 39.8 # 171
AP50 61.3 # 67
AP75 43.3 # 75
APS 22.9 # 63
APM 43.3 # 65
APL 52.6 # 65
Object Detection COCO test-dev Faster R-CNN + FPN box mAP 36.2 # 217
Hardware Burden 2G # 1
Pedestrian Detection TJU-Ped-campus FPN R (miss rate) 27.92 # 3
RS (miss rate) 67.52 # 1
HO (miss rate) 73.14 # 4
R+HO (miss rate) 35.67 # 3
ALL (miss rate) 38.08 # 3
Pedestrian Detection TJU-Ped-traffic FPN R (miss rate) 22.30 # 4
RS (miss rate) 35.19 # 2
HO (miss rate) 60.30 # 3
R+HO (miss rate) 26.71 # 3
ALL (miss rate) 37.78 # 3

Methods