Search Results for author:
Found 27 papers, 15 papers with code
Step-unrolled Denoising Autoencoders for Text Generation
In this paper we propose a new generative model of text, Step-unrolled Denoising Autoencoder (SUNDAE), that does not rely on autoregressive models.
Divide and Contrast: Self-supervised Learning from Uncurated Data
Self-supervised learning holds promise in leveraging large amounts of unlabeled data, however much of its progress has thus far been limited to highly curated pre-training data such as ImageNet.
Efficient Visual Pretraining with Contrastive Detection
Self-supervised pretraining has been shown to yield powerful representations for transfer learning.
Ranked #60 on
Semantic Segmentation
on Cityscapes val
(using extra training data)
Multi-Format Contrastive Learning of Audio Representations
Recent advances suggest the advantage of multi-modal training in comparison with single-modal methods.
Ranked #18 on
Audio Classification
on ESC-50
(using extra training data)
Self-supervised Adversarial Robustness for the Low-label, High-data Regime
Experiments on CIFAR-10 against $\ell_2$ and $\ell_\infty$ norm-bounded perturbations demonstrate that BYORL achieves near state-of-the-art robustness with as little as 500 labeled examples.
Learning Robust and Multilingual Speech Representations
Unsupervised speech representation learning has shown remarkable success at finding representations that correlate with phonetic structures and improve downstream speech recognition performance.
Unsupervised Learning of Efficient and Robust Speech Representations
We present an unsupervised method for learning speech representations based on a bidirectional contrastive predictive coding that implicitly discovers phonetic structure from large-scale corpora of unlabelled raw audio signals.
Shaping Belief States with Generative Environment Models for RL
We propose a way to efficiently train expressive generative models in complex environments.
Data-Efficient Image Recognition with Contrastive Predictive Coding
Human observers can learn to recognize new categories of images from a handful of examples, yet doing so with artificial ones remains an open challenge.
Ranked #6 on
Contrastive Learning
on imagenet-1k
On Variational Bounds of Mutual Information
Estimating and optimizing Mutual Information (MI) is core to many problems in machine learning; however, bounding MI in high dimensions is challenging.
Wasserstein Dependency Measure for Representation Learning
Mutual information maximization has emerged as a powerful learning objective for unsupervised representation learning obtaining state-of-the-art performance in applications such as object recognition, speech recognition, and reinforcement learning.
Representation Learning with Contrastive Predictive Coding
The key insight of our model is to learn such representations by predicting the future in latent space by using powerful autoregressive models.
Ranked #31 on
Semi-Supervised Image Classification
on ImageNet - 1% labeled data
(Top 5 Accuracy metric)
Associative Compression Networks for Representation Learning
We conclude that ACNs are a promising new direction for representation learning: one that steps away from IID modelling, and towards learning a structured description of the dataset as a whole.
Efficient Neural Audio Synthesis
The small number of weights in a Sparse WaveRNN makes it possible to sample high-fidelity audio on a mobile CPU in real time.
Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
We motivate 'adversarial risk' as an objective for achieving models robust to worst-case inputs.
Parallel WaveNet: Fast High-Fidelity Speech Synthesis
The recently-developed WaveNet architecture is the current state of the art in realistic speech synthesis, consistently rated as more natural sounding for many different languages than any previous system.
Neural Discrete Representation Learning
Learning useful representations without supervision remains a key challenge in machine learning.
Count-Based Exploration with Neural Density Models
This pseudo-count was used to generate an exploration bonus for a DQN agent and combined with a mixed Monte Carlo update was sufficient to achieve state of the art on the Atari 2600 game Montezuma's Revenge.
Ranked #9 on
Atari Games
on Atari 2600 Montezuma's Revenge
Neural Machine Translation in Linear Time
The ByteNet is a one-dimensional convolutional neural network that is composed of two parts, one to encode the source sequence and the other to decode the target sequence.
Ranked #1 on
Machine Translation
on WMT2015 English-German
Video Pixel Networks
The VPN approaches the best possible performance on the Moving MNIST benchmark, a leap over the previous state of the art, and the generated videos show only minor deviations from the ground truth.
Ranked #1 on
Video Prediction
on KTH
(Cond metric)
WaveNet: A Generative Model for Raw Audio
This paper introduces WaveNet, a deep neural network for generating raw audio waveforms.
Ranked #1 on
Speech Synthesis
on Mandarin Chinese
Conditional Image Generation with PixelCNN Decoders
This work explores conditional image generation with a new image density model based on the PixelCNN architecture.
Ranked #7 on
Density Estimation
on CIFAR-10
Pixel Recurrent Neural Networks
Modeling the distribution of natural images is a landmark problem in unsupervised learning.
Ranked #5 on
Image Generation
on Binarized MNIST
Factoring Variations in Natural Images with Deep Gaussian Mixture Models
In this paper we propose a new scalable deep generative model for images, called the Deep Gaussian Mixture Model, that is a straightforward but powerful generalization of GMMs to multiple layers.
Ranked #72 on
Image Generation
on CIFAR-10
(bits/dimension metric)
Deep content-based music recommendation
We also show that recent advances in deep learning translate very well to the music recommendation setting, with deep convolutional neural networks significantly outperforming the traditional approach.
