Rethinking Channel Dimensions for Efficient Model Design

Designing an efficient model within the limited computational cost is challenging. We argue the accuracy of a lightweight model has been further limited by the design convention: a stage-wise configuration of the channel dimensions, which looks like a piecewise linear function of the network stage. In this paper, we study an effective channel dimension configuration towards better performance than the convention. To this end, we empirically study how to design a single layer properly by analyzing the rank of the output feature. We then investigate the channel configuration of a model by searching network architectures concerning the channel configuration under the computational cost restriction. Based on the investigation, we propose a simple yet effective channel configuration that can be parameterized by the layer index. As a result, our proposed model following the channel parameterization achieves remarkable performance on ImageNet classification and transfer learning tasks including COCO object detection, COCO instance segmentation, and fine-grained classifications. Code and ImageNet pretrained models are available at https://github.com/clovaai/rexnet.

PDF Abstract CVPR 2021 PDF CVPR 2021 Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Image Classification ImageNet ReXNet_3.0 Top 1 Accuracy 82.8% # 244
Top 5 Accuracy 96.2% # 71
Number of params 34.7M # 199
Image Classification ImageNet ReXNet_0.6 Top 1 Accuracy 74.6% # 545
Top 5 Accuracy 92.1% # 203
Number of params 2.7M # 359
Image Classification ImageNet ReXNet_0.9 Top 1 Accuracy 77.2% # 478
Top 5 Accuracy 93.5% # 166
Number of params 4.1M # 349
Image Classification ImageNet ReXNet_1.5 Top 1 Accuracy 80.3% # 370
Top 5 Accuracy 95.2% # 103
Number of params 9.7M # 296
Image Classification ImageNet ReXNet_2.0 Top 1 Accuracy 81.6% # 320
Top 5 Accuracy 95.7% # 86
Number of params 19M # 267
Image Classification ImageNet ReXNet_1.3 Top 1 Accuracy 79.5% # 396
Top 5 Accuracy 94.7% # 123
Number of params 7.6M # 307
Image Classification ImageNet ReXNet_1.0 Top 1 Accuracy 77.9% # 466
Top 5 Accuracy 93.9% # 157
Number of params 4.8M # 341

Methods


No methods listed for this paper. Add relevant methods here