Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition

1 Feb 2021  ·  Ming Lin, Pichao Wang, Zhenhong Sun, Hesen Chen, Xiuyu Sun, Qi Qian, Hao Li, Rong Jin ·

Accuracy predictor is a key component in Neural Architecture Search (NAS) for ranking architectures. Building a high-quality accuracy predictor usually costs enormous computation. To address this issue, instead of using an accuracy predictor, we propose a novel zero-shot index dubbed Zen-Score to rank the architectures. The Zen-Score represents the network expressivity and positively correlates with the model accuracy. The calculation of Zen-Score only takes a few forward inferences through a randomly initialized network, without training network parameters. Built upon the Zen-Score, we further propose a new NAS algorithm, termed as Zen-NAS, by maximizing the Zen-Score of the target network under given inference budgets. Within less than half GPU day, Zen-NAS is able to directly search high performance architectures in a data-free style. Comparing with previous NAS methods, the proposed Zen-NAS is magnitude times faster on multiple server-side and mobile-side GPU platforms with state-of-the-art accuracy on ImageNet. Our source code and pre-trained models are released on https://github.com/idstcv/ZenNAS.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Neural Architecture Search CIFAR-10 ZenNet-2.0M Top-1 Error Rate 2.5% # 18
Parameters 2.0M # 17
FLOPS 487M # 35
Neural Architecture Search CIFAR-100 ZenNet-2.0M FLOPS 487M # 10
Percentage Error 15.6 # 7
PARAMS 2.0M # 4
Neural Architecture Search ImageNet ZenNAS (1.2ms) Top-1 Error Rate 16.4 # 1
Accuracy 83.6 # 1
FLOPs 22G # 1
Params 180M # 2
Image Classification ImageNet ZenNAS (0.8ms) Top 1 Accuracy 83.0% # 319
Number of params 183M # 712
GFLOPs 13.9 # 295
Neural Architecture Search ImageNet ZenNAS (0.1ms) Top-1 Error Rate 22.2 # 54
Accuracy 77.8 # 45
FLOPs 1.7G # 1
Params 30.1 # 57
Image Classification ImageNet ZenNet-400M-SE Top 1 Accuracy 78% # 617
Number of params 5.7M # 329
GFLOPs 0.820 # 90

Methods


No methods listed for this paper. Add relevant methods here