ICLR 2020

Meta-Learning without Memorization

ICLR 2020 google-research/google-research

If this is not done, the meta-learner can ignore the task training data and learn a single model that performs all of the meta-training tasks zero-shot, but does not adapt effectively to new image classes.

FEW-SHOT IMAGE CLASSIFICATION

Measuring Compositional Generalization: A Comprehensive Method on Realistic Data

ICLR 2020 google-research/google-research

We find that they fail to generalize compositionally and that there is a surprisingly strong negative correlation between compound divergence and accuracy.

QUESTION ANSWERING

Weakly Supervised Disentanglement with Guarantees

ICLR 2020 google-research/google-research

Learning disentangled representations that correspond to factors of variation in real-world data is critical to interpretable and human-controllable machine learning.

U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation

ICLR 2020 taki0112/UGATIT

We propose a novel method for unsupervised image-to-image translation, which incorporates a new attention module and a new learnable normalization function in an end-to-end manner.

UNSUPERVISED IMAGE-TO-IMAGE TRANSLATION

Reformer: The Efficient Transformer

ICLR 2020 google/trax

Large Transformer models routinely achieve state-of-the-art results on a number of tasks but training these models can be prohibitively costly, especially on long sequences.

LANGUAGE MODELLING

On the Variance of the Adaptive Learning Rate and Beyond

ICLR 2020 LiyuanLucasLiu/RAdam

The learning rate warmup heuristic achieves remarkable success in stabilizing training, accelerating convergence and improving generalization for adaptive stochastic optimization algorithms like RMSprop and Adam.

IMAGE CLASSIFICATION LANGUAGE MODELLING MACHINE TRANSLATION STOCHASTIC OPTIMIZATION

DiffTaichi: Differentiable Programming for Physical Simulation

ICLR 2020 yuanming-hu/difftaichi

We present DiffTaichi, a new differentiable programming language tailored for building high-performance differentiable physical simulators.

PHYSICAL SIMULATIONS

DDSP: Differentiable Digital Signal Processing

ICLR 2020 magenta/ddsp

In this paper, we introduce the Differentiable Digital Signal Processing (DDSP) library, which enables direct integration of classic signal processing elements with deep learning methods.

AUDIO GENERATION

ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators

ICLR 2020 google-research/electra

Then, instead of training a model that predicts the original identities of the corrupted tokens, we train a discriminative model that predicts whether each token in the corrupted input was replaced by a generator sample or not.

LANGUAGE MODELLING NATURAL LANGUAGE UNDERSTANDING