sharpDARTS: Faster and More Accurate Differentiable Architecture Search

23 Mar 2019 Andrew Hundt Varun Jain Gregory D. Hager

Neural Architecture Search (NAS) has been a source of dramatic improvements in neural network design, with recent results meeting or exceeding the performance of hand-tuned architectures. However, our understanding of how to represent the search space for neural net architectures and how to search that space efficiently are both still in their infancy... (read more)

PDF Abstract
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Neural Architecture Search CIFAR-10 sharpDARTS Top-1 Error Rate 2.29% # 3
Search Time (GPU days) 1.8 # 12
Parameters 1.98M # 9
FLOPS 357M # 12
Neural Architecture Search CIFAR-10 SharpSepConvDARTS Top-1 Error Rate 1.98% # 1
Search Time (GPU days) 0.8 # 7
Parameters 3.6M # 13
FLOPS 579M # 17
Neural Architecture Search CIFAR-10 Image Classification SharpSepConvDARTS Percentage error 1.98 # 4
Params 3.6M # 5
FLOPS 579M # 16
Neural Architecture Search ImageNet SharpSepConvDARTS Top-1 Error Rate 25.1 # 66
Accuracy 74.1 # 64
Params 4.9M # 25
MACs 573M # 64
Neural Architecture Search ImageNet sharpDARTS Top-1 Error Rate 24.0 # 54
Accuracy 76.0 # 54
Params 8.3M # 4
MACs 950M # 73

Methods used in the Paper


METHOD TYPE
Exponential Decay
Learning Rate Schedules
DARTS Max-W
Neural Architecture Search
Differentiable Hyperparameter Search
Hyperparameter Search
DARTS
Neural Architecture Search
Cosine Power Annealing
Learning Rate Schedules