Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks

18 Dec 2015  ·  Li Shen, Zhouchen Lin, Qingming Huang ·

Learning deeper convolutional neural networks becomes a tendency in recent years. However, many empirical evidences suggest that performance improvement cannot be gained by simply stacking more layers. In this paper, we consider the issue from an information theoretical perspective, and propose a novel method Relay Backpropagation, that encourages the propagation of effective information through the network in training stage. By virtue of the method, we achieved the first place in ILSVRC 2015 Scene Classification Challenge. Extensive experiments on two challenging large scale datasets demonstrate the effectiveness of our method is not restricted to a specific dataset or network architecture. Our models will be available to the research community later.

PDF Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Long-tail Learning COCO-MLT RS(ResNet-50) Average mAP 46.97 # 10
Long-tail Learning VOC-MLT RS(ResNet-50) Average mAP 75.38 # 8

Methods


No methods listed for this paper. Add relevant methods here