Browse > Methodology > Architecture Search

Architecture Search

29 papers with code · Methodology

State-of-the-art leaderboards

No evaluation results yet. Help compare methods by submit evaluation metrics.

Greatest papers with code

Progressive Neural Architecture Search

ECCV 2018 tensorflow/models

We propose a new method for learning the structure of convolutional neural networks (CNNs) that is more efficient than recent state-of-the-art methods based on reinforcement learning and evolutionary algorithms. Our approach uses a sequential model-based optimization (SMBO) strategy, in which we search for structures in order of increasing complexity, while simultaneously learning a surrogate model to guide the search through structure space.

ARCHITECTURE SEARCH

Learning Transferable Architectures for Scalable Image Recognition

CVPR 2018 tensorflow/models

In our experiments, we search for the best convolutional layer (or "cell") on the CIFAR-10 dataset and then apply this cell to the ImageNet dataset by stacking together more copies of this cell, each with their own parameters to design a convolutional architecture, named "NASNet architecture". For instance, a small version of NASNet also achieves 74% top-1 accuracy, which is 3.1% better than equivalently-sized, state-of-the-art models for mobile platforms.

ARCHITECTURE SEARCH IMAGE CLASSIFICATION

The Evolved Transformer

30 Jan 2019tensorflow/tensor2tensor

Recent works have highlighted the strengths of the Transformer architecture for dealing with sequence tasks. At the same time, neural architecture search has advanced to the point where it can outperform human-designed models.

ARCHITECTURE SEARCH MACHINE TRANSLATION

Auto-Keras: Efficient Neural Architecture Search with Network Morphism

27 Jun 2018jhfjhfj1/autokeras

Neural architecture search (NAS) has been proposed to automatically tune deep neural networks, but existing search algorithms usually suffer from expensive computational cost. Network morphism, which keeps the functionality of a neural network while changing its neural architecture, could be helpful for NAS by enabling a more efficient training during the search.

ARCHITECTURE SEARCH

DARTS: Differentiable Architecture Search

ICLR 2019 quark0/darts

This paper addresses the scalability challenge of architecture search by formulating the task in a differentiable manner. Unlike conventional approaches of applying evolution or reinforcement learning over a discrete and non-differentiable search space, our method is based on the continuous relaxation of the architecture representation, allowing efficient search of the architecture using gradient descent.

ARCHITECTURE SEARCH IMAGE CLASSIFICATION LANGUAGE MODELLING

Efficient Neural Architecture Search via Parameter Sharing

9 Feb 2018melodyguan/enas

The controller is trained with policy gradient to select a subgraph that maximizes the expected reward on the validation set. On the CIFAR-10 dataset, ENAS designs novel architectures that achieve a test error of 2.89%, which is on par with NASNet (Zoph et al., 2018), whose test error is 2.65%.

ARCHITECTURE SEARCH LANGUAGE MODELLING

SMASH: One-Shot Model Architecture Search through HyperNetworks

ICLR 2018 ajbrock/SMASH

Designing architectures for deep neural networks requires expert knowledge and substantial computation time. We propose a technique to accelerate architecture selection by learning an auxiliary HyperNet that generates the weights of a main model conditioned on that model's architecture.

ARCHITECTURE SEARCH

Rethinking the Value of Network Pruning

ICLR 2019 Eric-mingjie/rethinking-network-pruning

Network pruning is widely used for reducing the heavy computational cost of deep models. Our results have several implications: 1) training a large, over-parameterized model is not necessary to obtain an efficient final model, 2) learned "important" weights of the large model are not necessarily useful for the small pruned model, 3) the pruned architecture itself, rather than a set of inherited "important" weights, is what leads to the efficiency benefit in the final model, which suggests that some pruning algorithms could be seen as performing network architecture search.

ARCHITECTURE SEARCH NETWORK PRUNING

ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware

ICLR 2019 MIT-HAN-LAB/ProxylessNAS

In this paper, we present \emph{ProxylessNAS} that can \emph{directly} learn the architectures for large-scale target tasks and target hardware platforms. We address the high memory consumption issue of differentiable NAS and reduce the computational cost (GPU hours and GPU memory) to the same level of regular training while still allowing a large candidate set.

ARCHITECTURE SEARCH

Neural Architecture Optimization

NeurIPS 2018 renqianluo/NAO

The performance predictor and the encoder enable us to perform gradient based optimization in the continuous space to find the embedding of a new architecture with potentially better accuracy. Experiments show that the architecture discovered by our method is very competitive for image classification task on CIFAR-10 and language modeling task on PTB, outperforming or on par with the best results of previous architecture search methods with a significantly reduction of computational resources.

ARCHITECTURE SEARCH IMAGE CLASSIFICATION LANGUAGE MODELLING