DARTS: Differentiable Architecture Search

ICLR 2019 Hanxiao LiuKaren SimonyanYiming Yang

This paper addresses the scalability challenge of architecture search by formulating the task in a differentiable manner. Unlike conventional approaches of applying evolution or reinforcement learning over a discrete and non-differentiable search space, our method is based on the continuous relaxation of the architecture representation, allowing efficient search of the architecture using gradient descent... (read more)

PDF Abstract
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT LEADERBOARD
Neural Architecture Search ImageNet DARTS Top-1 Error Rate 26.7% # 22
Language Modelling Penn Treebank (Word Level) Differentiable NAS Validation perplexity 58.3 # 17
Test perplexity 56.1 # 21
Params 23M # 1