Search Results for author:
Found 35 papers, 23 papers with code
Bayesian Flow Networks
Notably, the network inputs for discrete data lie on the probability simplex, and are therefore natively differentiable, paving the way for gradient-based sample guidance and few-step generation in discrete domains such as language modelling.
Ranked #3 on
Image Generation
on Binarized MNIST
Practical Real Time Recurrent Learning with a Sparse Approximation
For highly sparse networks, SnAp with $n=2$ remains tractable and can outperform backpropagation through time in terms of learning speed when updates are done online.
A Practical Sparse Approximation for Real Time Recurrent Learning
Current methods for training recurrent neural networks are based on backpropagation through time, which requires storing a complete history of network states, and prohibits updating the weights `online' (after every timestep).
Associative Compression Networks for Representation Learning
We conclude that ACNs are a promising new direction for representation learning: one that steps away from IID modelling, and towards learning a structured description of the dataset as a whole.
The Kanerva Machine: A Generative Distributed Memory
We present an end-to-end trained memory system that quickly adapts to new data and generates samples like them.
Parallel WaveNet: Fast High-Fidelity Speech Synthesis
The recently-developed WaveNet architecture is the current state of the art in realistic speech synthesis, consistently rated as more natural sounding for many different languages than any previous system.
Noisy Networks for Exploration
We introduce NoisyNet, a deep reinforcement learning agent with parametric noise added to its weights, and show that the induced stochasticity of the agent's policy can be used to aid efficient exploration.
Ranked #1 on
Atari Games
on Atari 2600 Surround
Automated Curriculum Learning for Neural Networks
We introduce a method for automatically selecting the path, or syllabus, that a neural network follows through a curriculum so as to maximise learning efficiency.
Neural Machine Translation in Linear Time
The ByteNet is a one-dimensional convolutional neural network that is composed of two parts, one to encode the source sequence and the other to decode the target sequence.
Ranked #1 on
Machine Translation
on WMT2015 English-German
Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes
SAM learns with comparable data efficiency to existing models on a range of synthetic tasks and one-shot Omniglot character recognition, and can scale to tasks requiring $100,\! 000$s of time steps and memories.
Ranked #6 on
Question Answering
on bAbi
(Mean Error Rate metric)
Video Pixel Networks
The VPN approaches the best possible performance on the Moving MNIST benchmark, a leap over the previous state of the art, and the generated videos show only minor deviations from the ground truth.
Ranked #1 on
Video Prediction
on KTH
(Cond metric)
WaveNet: A Generative Model for Raw Audio
This paper introduces WaveNet, a deep neural network for generating raw audio waveforms.
Ranked #1 on
Speech Synthesis
on Mandarin Chinese
Decoupled Neural Interfaces using Synthetic Gradients
Training directed neural networks typically requires forward-propagating data through a computation graph, followed by backpropagating error signal, to produce weight updates.
Stochastic Backpropagation through Mixture Density Distributions
This report describes an alternative transform, applicable to any continuous multivariate distribution with a differentiable density function from which samples can be drawn, and uses it to derive an unbiased estimator for mixture density weight derivatives.
Conditional Image Generation with PixelCNN Decoders
This work explores conditional image generation with a new image density model based on the PixelCNN architecture.
Ranked #7 on
Density Estimation
on CIFAR-10
Strategic Attentive Writer for Learning Macro-Actions
We present a novel deep recurrent neural network architecture that learns to build implicit plans in an end-to-end manner by purely interacting with an environment in reinforcement learning setting.
Memory-Efficient Backpropagation Through Time
We propose a novel approach to reduce memory consumption of the backpropagation through time (BPTT) algorithm when training recurrent neural networks (RNNs).
Adaptive Computation Time for Recurrent Neural Networks
This paper introduces Adaptive Computation Time (ACT), an algorithm that allows recurrent neural networks to learn how many computational steps to take between receiving an input and emitting an output.
Associative Long Short-Term Memory
We investigate a new method to augment recurrent neural networks with extra memory without increasing the number of network parameters.
Asynchronous Methods for Deep Reinforcement Learning
We propose a conceptually simple and lightweight framework for deep reinforcement learning that uses asynchronous gradient descent for optimization of deep neural network controllers.
Ranked #9 on
Atari Games
on Atari 2600 Star Gunner
Grid Long Short-Term Memory
This paper introduces Grid Long Short-Term Memory, a network of LSTM cells arranged in a multidimensional grid that can be applied to vectors, sequences or higher dimensional data such as images.
Human level control through deep reinforcement learning
We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester across a set of 49 games, using the same algorithm, network architecture and hyperparameters.
DRAW: A Recurrent Neural Network For Image Generation
This paper introduces the Deep Recurrent Attentive Writer (DRAW) neural network architecture for image generation.
Ranked #70 on
Image Generation
on CIFAR-10
(bits/dimension metric)
Neural Turing Machines
We extend the capabilities of neural networks by coupling them to external memory resources, which they can interact with by attentional processes.
Recurrent Models of Visual Attention
Applying convolutional neural networks to large images is computationally expensive because the amount of computation scales linearly with the number of image pixels.
Towards End-To-End Speech Recognition with Recurrent Neural Networks
A modification to the objective function is introduced that trains the network to minimise the expectation of an arbitrary transcription loss function.
Playing Atari with Deep Reinforcement Learning
We present the first deep learning model to successfully learn control policies directly from high-dimensional sensory input using reinforcement learning.
Ranked #1 on
Atari Games
on Atari 2600 Pong
Generating Sequences With Recurrent Neural Networks
This paper shows how Long Short-term Memory recurrent neural networks can be used to generate complex sequences with long-range structure, simply by predicting one data point at a time.
Ranked #41 on
Language Modelling
on enwik8
Speech Recognition with Deep Recurrent Neural Networks
Recurrent neural networks (RNNs) are a powerful model for sequential data.
Ranked #18 on
Speech Recognition
on TIMIT
Sequence Transduction with Recurrent Neural Networks
One of the key challenges in sequence transduction is learning to represent both the input and output sequences in a way that is invariant to sequential distortions such as shrinking, stretching and translating.
Practical Variational Inference for Neural Networks
Variational methods have been previously explored as a tractable approximation to Bayesian inference for neural networks.
Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks
Offline handwriting recognition---the transcription of images of handwritten text---is an interesting task, in that it combines computer vision with sequence learning.
Unconstrained On-line Handwriting Recognition with Recurrent Neural Networks
On-line handwriting recognition is unusual among sequence labelling tasks in that the underlying generator of the observed data, i. e. the movement of the pen, is recorded directly.
Multi-Dimensional Recurrent Neural Networks
Recurrent neural networks (RNNs) have proved effective at one dimensional sequence learning tasks, such as speech and online handwriting recognition.
Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks
Many real-world sequence learning tasks require the prediction of sequences of labels from noisy, unsegmented input data.
