Efficient Neural Architecture Search via Parameter Sharing

9 Feb 2018 Hieu Pham Melody Y. Guan Barret Zoph Quoc V. Le Jeff Dean

We propose Efficient Neural Architecture Search (ENAS), a fast and inexpensive approach for automatic model design. In ENAS, a controller learns to discover neural network architectures by searching for an optimal subgraph within a large computational graph... (read more)

PDF Abstract

Datasets


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Neural Architecture Search CIFAR-10 Image Classification ENAS + c/o Percentage error 2.89 # 15
Params 4.6M # 8
Language Modelling Penn Treebank (Word Level) Efficient NAS Validation perplexity 60.8 # 23
Test perplexity 58.6 # 29
Params 24M # 7

Methods used in the Paper


METHOD TYPE
Sigmoid Activation
Activation Functions
Tanh Activation
Activation Functions
Softmax
Output Functions
LSTM
Recurrent Neural Networks