ICML 2017

Learned Optimizers that Scale and Generalize

ICML 2017 tensorflow/models

Two of the primary barriers to its adoption are an inability to scale to larger problems and a limited ability to generalize to new tasks.

Convolutional Sequence to Sequence Learning

ICML 2017 facebookresearch/fairseq-py

The prevalent approach to sequence to sequence learning maps an input sequence to a variable length output sequence via recurrent neural networks.

MACHINE TRANSLATION

Language Modeling with Gated Convolutional Networks

ICML 2017 facebookresearch/fairseq-py

The pre-dominant approach to language modeling to date is based on recurrent neural networks.

LANGUAGE MODELLING

Image-to-Markup Generation with Coarse-to-Fine Attention

ICML 2017 da03/Attention-OCR

We present a neural encoder-decoder model to convert images into presentational markup based on a scalable coarse-to-fine attention mechanism.

OPTICAL CHARACTER RECOGNITION

Deep Voice: Real-time Neural Text-to-Speech

ICML 2017 NVIDIA/nv-wavenet

We present Deep Voice, a production-quality text-to-speech system constructed entirely from deep neural networks.

BOUNDARY DETECTION SPEECH SYNTHESIS

Neural Message Passing for Quantum Chemistry

ICML 2017 Microsoft/gated-graph-neural-network-samples

Supervised learning on molecules has incredible potential to be useful in chemistry, drug discovery, and materials science.

DRUG DISCOVERY

Recurrent Highway Networks

ICML 2017 julian121266/RecurrentHighwayNetworks

We introduce a novel theoretical analysis of recurrent networks based on Gersgorin's circle theorem that illuminates several modeling and optimization issues and improves our understanding of the LSTM cell.

LANGUAGE MODELLING

Efficient softmax approximation for GPUs

ICML 2017 facebookresearch/adaptive-softmax

We propose an approximate strategy to efficiently train neural network based language models over very large vocabularies.

Developing Bug-Free Machine Learning Systems With Formal Mathematics

ICML 2017 dselsam/certigrad

As a case study, we implement a new system, Certigrad, for optimizing over stochastic computation graphs, and we generate a formal (i.e. machine-checkable) proof that the gradients sampled by the system are unbiased estimates of the true mathematical gradients.

DeepBach: a Steerable Model for Bach Chorales Generation

ICML 2017 Ghadjeres/DeepBach

This paper introduces DeepBach, a graphical model aimed at modeling polyphonic music and specifically hymn-like pieces.