ICLR 2019

Meta-Learning Update Rules for Unsupervised Representation Learning

ICLR 2019 tensorflow/models

Specifically, we target semi-supervised classification performance, and we meta-learn an algorithm -- an unsupervised weight update rule -- that produces representations useful for this task.

META-LEARNING UNSUPERVISED REPRESENTATION LEARNING

GANSynth: Adversarial Neural Audio Synthesis

ICLR 2019 tensorflow/magenta

Efficient audio synthesis is an inherently difficult machine learning task, as human perception is sensitive to both global structure and fine-scale waveform coherence.

AUDIO GENERATION

Universal Transformers

ICLR 2019 tensorflow/tensor2tensor

Feed-forward and convolutional architectures have recently been shown to achieve superior results on some sequence modeling tasks such as machine translation, with the added advantage that they concurrently process all inputs in the sequence, leading to easy parallelization and faster training times.

LANGUAGE MODELLING LEARNING TO EXECUTE MACHINE TRANSLATION

Pay Less Attention with Lightweight and Dynamic Convolutions

ICLR 2019 pytorch/fairseq

We predict separate convolution kernels based solely on the current time-step in order to determine the importance of context elements.

ABSTRACTIVE TEXT SUMMARIZATION LANGUAGE MODELLING MACHINE TRANSLATION

Adaptive Input Representations for Neural Language Modeling

ICLR 2019 pytorch/fairseq

We introduce adaptive input representations for neural language modeling which extend the adaptive softmax of Grave et al. (2017) to input representations of variable capacity.

LANGUAGE MODELLING

Wizard of Wikipedia: Knowledge-Powered Conversational agents

ICLR 2019 facebookresearch/ParlAI

In open-domain dialogue intelligent agents should exhibit the use of knowledge, however there are few convincing demonstrations of this to date.

Adaptive Gradient Methods with Dynamic Bound of Learning Rate

ICLR 2019 Luolc/AdaBound

Adaptive optimization methods such as AdaGrad, RMSProp and Adam have been proposed to achieve a rapid training process with an element-wise scaling term on learning rates.

STOCHASTIC OPTIMIZATION

DARTS: Differentiable Architecture Search

ICLR 2019 quark0/darts

This paper addresses the scalability challenge of architecture search by formulating the task in a differentiable manner.

IMAGE CLASSIFICATION LANGUAGE MODELLING NEURAL ARCHITECTURE SEARCH