Search Results for author: Aaron van den Oord

Found 27 papers, 14 papers with code

Step-unrolled Denoising Autoencoders for Text Generation

1 code implementation ICLR 2022 Nikolay Savinov, Junyoung Chung, Mikolaj Binkowski, Erich Elsen, Aaron van den Oord

In this paper we propose a new generative model of text, Step-unrolled Denoising Autoencoder (SUNDAE), that does not rely on autoregressive models.

Denoising Language Modelling +2

Divide and Contrast: Self-supervised Learning from Uncurated Data

no code implementations ICCV 2021 Yonglong Tian, Olivier J. Henaff, Aaron van den Oord

Self-supervised learning holds promise in leveraging large amounts of unlabeled data, however much of its progress has thus far been limited to highly curated pre-training data such as ImageNet.

Contrastive Learning Self-Supervised Image Classification +1

Multi-Format Contrastive Learning of Audio Representations

no code implementations11 Mar 2021 Luyu Wang, Aaron van den Oord

Recent advances suggest the advantage of multi-modal training in comparison with single-modal methods.

Ranked #12 on Audio Classification on ESC-50 (using extra training data)

Audio Classification Contrastive Learning

Self-supervised Adversarial Robustness for the Low-label, High-data Regime

no code implementations ICLR 2021 Sven Gowal, Po-Sen Huang, Aaron van den Oord, Timothy Mann, Pushmeet Kohli

Experiments on CIFAR-10 against $\ell_2$ and $\ell_\infty$ norm-bounded perturbations demonstrate that BYORL achieves near state-of-the-art robustness with as little as 500 labeled examples.

Adversarial Robustness Self-Supervised Learning +1

Learning Robust and Multilingual Speech Representations

no code implementations Findings of the Association for Computational Linguistics 2020 Kazuya Kawakami, Luyu Wang, Chris Dyer, Phil Blunsom, Aaron van den Oord

Unsupervised speech representation learning has shown remarkable success at finding representations that correlate with phonetic structures and improve downstream speech recognition performance.

Representation Learning speech-recognition +1

Unsupervised Learning of Efficient and Robust Speech Representations

no code implementations25 Sep 2019 Kazuya Kawakami, Luyu Wang, Chris Dyer, Phil Blunsom, Aaron van den Oord

We present an unsupervised method for learning speech representations based on a bidirectional contrastive predictive coding that implicitly discovers phonetic structure from large-scale corpora of unlabelled raw audio signals.

speech-recognition Speech Recognition

On Variational Bounds of Mutual Information

3 code implementations16 May 2019 Ben Poole, Sherjil Ozair, Aaron van den Oord, Alexander A. Alemi, George Tucker

Estimating and optimizing Mutual Information (MI) is core to many problems in machine learning; however, bounding MI in high dimensions is challenging.

Representation Learning

Wasserstein Dependency Measure for Representation Learning

no code implementations NeurIPS 2019 Sherjil Ozair, Corey Lynch, Yoshua Bengio, Aaron van den Oord, Sergey Levine, Pierre Sermanet

Mutual information maximization has emerged as a powerful learning objective for unsupervised representation learning obtaining state-of-the-art performance in applications such as object recognition, speech recognition, and reinforcement learning.

Object Recognition reinforcement-learning +5

Representation Learning with Contrastive Predictive Coding

27 code implementations10 Jul 2018 Aaron van den Oord, Yazhe Li, Oriol Vinyals

The key insight of our model is to learn such representations by predicting the future in latent space by using powerful autoregressive models.

Representation Learning Self-Supervised Image Classification +1

Associative Compression Networks for Representation Learning

no code implementations6 Apr 2018 Alex Graves, Jacob Menick, Aaron van den Oord

We conclude that ACNs are a promising new direction for representation learning: one that steps away from IID modelling, and towards learning a structured description of the dataset as a whole.

Representation Learning

Neural Discrete Representation Learning

43 code implementations NeurIPS 2017 Aaron van den Oord, Oriol Vinyals, Koray Kavukcuoglu

Learning useful representations without supervision remains a key challenge in machine learning.

Representation Learning

Count-Based Exploration with Neural Density Models

no code implementations ICML 2017 Georg Ostrovski, Marc G. Bellemare, Aaron van den Oord, Remi Munos

This pseudo-count was used to generate an exploration bonus for a DQN agent and combined with a mixed Monte Carlo update was sufficient to achieve state of the art on the Atari 2600 game Montezuma's Revenge.

Montezuma's Revenge

Neural Machine Translation in Linear Time

10 code implementations31 Oct 2016 Nal Kalchbrenner, Lasse Espeholt, Karen Simonyan, Aaron van den Oord, Alex Graves, Koray Kavukcuoglu

The ByteNet is a one-dimensional convolutional neural network that is composed of two parts, one to encode the source sequence and the other to decode the target sequence.

Language Modelling Machine Translation +2

Video Pixel Networks

1 code implementation ICML 2017 Nal Kalchbrenner, Aaron van den Oord, Karen Simonyan, Ivo Danihelka, Oriol Vinyals, Alex Graves, Koray Kavukcuoglu

The VPN approaches the best possible performance on the Moving MNIST benchmark, a leap over the previous state of the art, and the generated videos show only minor deviations from the ground truth.

 Ranked #1 on Video Prediction on KTH (Cond metric)

Video Prediction

Pixel Recurrent Neural Networks

17 code implementations25 Jan 2016 Aaron van den Oord, Nal Kalchbrenner, Koray Kavukcuoglu

Modeling the distribution of natural images is a landmark problem in unsupervised learning.

Image Generation

Factoring Variations in Natural Images with Deep Gaussian Mixture Models

no code implementations NeurIPS 2014 Aaron Van Den Oord, Benjamin Schrauwen

In this paper we propose a new scalable deep generative model for images, called the Deep Gaussian Mixture Model, that is a straightforward but powerful generalization of GMMs to multiple layers.

Ranked #64 on Image Generation on CIFAR-10 (bits/dimension metric)

Density Estimation Image Generation +1

Deep content-based music recommendation

no code implementations NeurIPS 2013 Aaron Van Den Oord, Sander Dieleman, Benjamin Schrauwen

We also show that recent advances in deep learning translate very well to the music recommendation setting, with deep convolutional neural networks significantly outperforming the traditional approach.

Collaborative Filtering Music Recommendation +1

Cannot find the paper you are looking for? You can Submit a new open access paper.