Deformable Convolutional Networks

Convolutional neural networks (CNNs) are inherently limited to model geometric transformations due to the fixed geometric structures in its building modules. In this work, we introduce two new modules to enhance the transformation modeling capacity of CNNs, namely, deformable convolution and deformable RoI pooling. Both are based on the idea of augmenting the spatial sampling locations in the modules with additional offsets and learning the offsets from target tasks, without additional supervision. The new modules can readily replace their plain counterparts in existing CNNs and can be easily trained end-to-end by standard back-propagation, giving rise to deformable convolutional networks. Extensive experiments validate the effectiveness of our approach on sophisticated vision tasks of object detection and semantic segmentation. The code would be released.

PDF Abstract ICCV 2017 PDF ICCV 2017 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Object Detection COCO test-dev DeformConv-R-FCN (Aligned-Inception-ResNet) box mAP 37.5 # 211
AP50 58.0 # 146
APS 19.4 # 136
APM 40.1 # 138
APL 52.5 # 122
Hardware Burden None # 1
Operations per network pass None # 1
Vessel Detection Vessel detection Dateset Deformable DETR AP 54.8% # 3

Methods