Gated Convolutional Networks with Hybrid Connectivity for Image Classification

26 Aug 2019  ·  Chuanguang Yang, Zhulin An, Hui Zhu, Xiaolong Hu, Kun Zhang, Kaiqiang Xu, Chao Li, Yongjun Xu ·

We propose a simple yet effective method to reduce the redundancy of DenseNet by substantially decreasing the number of stacked modules by replacing the original bottleneck by our SMG module, which is augmented by local residual. Furthermore, SMG module is equipped with an efficient two-stage pipeline, which aims to DenseNet-like architectures that need to integrate all previous outputs, i.e., squeezing the incoming informative but redundant features gradually by hierarchical convolutions as a hourglass shape and then exciting it by multi-kernel depthwise convolutions, the output of which would be compact and hold more informative multi-scale features. We further develop a forget and an update gate by introducing the popular attention modules to implement the effective fusion instead of a simple addition between reused and new features. Due to the Hybrid Connectivity (nested combination of global dense and local residual) and Gated mechanisms, we called our network as the HCGNet. Experimental results on CIFAR and ImageNet datasets show that HCGNet is more prominently efficient than DenseNet, and can also significantly outperform state-of-the-art networks with less complexity. Moreover, HCGNet also shows the remarkable interpretability and robustness by network dissection and adversarial defense, respectively. On MS-COCO, HCGNet can consistently learn better features than popular backbones.

PDF Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Image Classification CIFAR-10 HCGNet-A2 Percentage correct 97.71 # 67
PARAMS 3.1M # 189
Image Classification CIFAR-10 HCGNet-A1 Percentage correct 96.85 # 92
PARAMS 1.1M # 184
Image Classification CIFAR-10 HCGNet-A3 Percentage correct 97.86 # 60
PARAMS 11.4M # 200
Image Classification CIFAR-100 HCGNet-A2 Percentage correct 83.46 # 87
PARAMS 3.1M # 182
Image Classification CIFAR-100 HCGNet-A3 Percentage correct 84.04 # 80
PARAMS 11.4M # 189
Image Classification CIFAR-100 HCGNet-A1 Percentage correct 81.87 # 110
PARAMS 1.1M # 180
Image Classification ImageNet HCGNet-B Top 1 Accuracy 78.5% # 757
Number of params 12.9M # 504
GFLOPs 2.0 # 146
Image Classification ImageNet HCGNet-C Top 1 Accuracy 80.5% # 635
Number of params 42.2M # 688
GFLOPs 7.1 # 250