Search Results for author:
Found 52 papers, 39 papers with code
Improving alignment of dialogue agents via targeted human judgements
We present Sparrow, an information-seeking dialogue agent trained to be more helpful, correct, and harmless compared to prompted language model baselines.
Competition-Level Code Generation with AlphaCode
Programming is a powerful and ubiquitous problem-solving tool.
Ranked #1 on
Code Generation
on CodeContests
Unified Scaling Laws for Routed Language Models
The performance of a language model has been shown to be effectively modeled as a power-law in its parameter count.
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.
Ranked #1 on
College Mathematics
on BIG-bench
(using extra training data)
Highly accurate protein structure prediction with AlphaFold
Accurate computational approaches are needed to address this gap and to enable large-scale structural bioinformatics.
Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning
From an augmented view of an image, we train the online network to predict the target network representation of the same image under a different augmented view.
Bootstrap your own latent: A new approach to self-supervised Learning
From an augmented view of an image, we train the online network to predict the target network representation of the same image under a different augmented view.
Ranked #2 on
Self-Supervised Person Re-Identification
on SYSU-30k
Representation Learning
Self-Supervised Image Classification
+3
The StreetLearn Environment and Dataset
These datasets cannot be used for decision-making and reinforcement learning, however, and in general the perspective of navigation as an interactive learning task, where the actions and behaviours of a learning agent are learned simultaneously with the perception and planning, is relatively unsupported.
Human-level performance in first-person multiplayer games with population-based deep reinforcement learning
Recent progress in artificial intelligence through reinforcement learning (RL) has shown great success on increasingly complex single-agent environments and two-player turn-based games.
Learning to Navigate in Cities Without a Map
We present an interactive navigation environment that uses Google StreetView for its photographic content and worldwide coverage, and demonstrate that our learning method allows agents to learn to navigate multiple cities and to traverse to target destinations that may be kilometres away.
Unsupervised Predictive Memory in a Goal-Directed Agent
Animals execute goal-directed behaviours despite the limited range and scope of their sensors.
Efficient Neural Audio Synthesis
The small number of weights in a Sparse WaveRNN makes it possible to sample high-fidelity audio on a mobile CPU in real time.
IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
In this work we aim to solve a large collection of tasks using a single reinforcement learning agent with a single set of parameters.
Ranked #3 on
Atari Games
on Atari 2600 Skiing
(using extra training data)
Parallel WaveNet: Fast High-Fidelity Speech Synthesis
The recently-developed WaveNet architecture is the current state of the art in realistic speech synthesis, consistently rated as more natural sounding for many different languages than any previous system.
Population Based Training of Neural Networks
Neural networks dominate the modern machine learning landscape, but their training and success still suffer from sensitivity to empirical choices of hyperparameters such as model architecture, loss function, and optimisation algorithm.
Neural Discrete Representation Learning
Learning useful representations without supervision remains a key challenge in machine learning.
Hierarchical Representations for Efficient Architecture Search
We explore efficient neural architecture search methods and show that a simple yet powerful evolutionary algorithm can discover new architectures with excellent performance.
Automated Curriculum Learning for Neural Networks
We introduce a method for automatically selecting the path, or syllabus, that a neural network follows through a curriculum so as to maximise learning efficiency.
FeUdal Networks for Hierarchical Reinforcement Learning
We introduce FeUdal Networks (FuNs): a novel architecture for hierarchical reinforcement learning.
Hierarchical Reinforcement Learning
reinforcement-learning
+1
Understanding Synthetic Gradients and Decoupled Neural Interfaces
When training neural networks, the use of Synthetic Gradients (SG) allows layers or modules to be trained without update locking - without waiting for a true error gradient to be backpropagated - resulting in Decoupled Neural Interfaces (DNIs).
Interaction Networks for Learning about Objects, Relations and Physics
Here we introduce the interaction network, a model which can reason about how objects in complex systems interact, supporting dynamical predictions, as well as inferences about the abstract properties of the system.
Reinforcement Learning with Unsupervised Auxiliary Tasks
We also introduce a novel mechanism for focusing this representation upon extrinsic rewards, so that learning can rapidly adapt to the most relevant aspects of the actual task.
Learning to Navigate in Complex Environments
Learning to navigate in complex environments with dynamic elements is an important milestone in developing AI agents.
Combining policy gradient and Q-learning
Policy gradient is an efficient technique for improving a policy in a reinforcement learning setting.
Sample Efficient Actor-Critic with Experience Replay
This paper presents an actor-critic deep reinforcement learning agent with experience replay that is stable, sample efficient, and performs remarkably well on challenging environments, including the discrete 57-game Atari domain and several continuous control problems.
Neural Machine Translation in Linear Time
The ByteNet is a one-dimensional convolutional neural network that is composed of two parts, one to encode the source sequence and the other to decode the target sequence.
Ranked #1 on
Machine Translation
on WMT2015 English-German
Video Pixel Networks
The VPN approaches the best possible performance on the Moving MNIST benchmark, a leap over the previous state of the art, and the generated videos show only minor deviations from the ground truth.
Ranked #1 on
Video Prediction
on KTH
(Cond metric)
WaveNet: A Generative Model for Raw Audio
This paper introduces WaveNet, a deep neural network for generating raw audio waveforms.
Ranked #1 on
Speech Synthesis
on Mandarin Chinese
Decoupled Neural Interfaces using Synthetic Gradients
Training directed neural networks typically requires forward-propagating data through a computation graph, followed by backpropagating error signal, to produce weight updates.
Conditional Image Generation with PixelCNN Decoders
This work explores conditional image generation with a new image density model based on the PixelCNN architecture.
Ranked #10 on
Image Generation
on ImageNet 32x32
(bpd metric)
Strategic Attentive Writer for Learning Macro-Actions
We present a novel deep recurrent neural network architecture that learns to build implicit plans in an end-to-end manner by purely interacting with an environment in reinforcement learning setting.
Progressive Neural Networks
Learning to solve complex sequences of tasks--while both leveraging transfer and avoiding catastrophic forgetting--remains a key obstacle to achieving human-level intelligence.
Matching Networks for One Shot Learning
Our algorithm improves one-shot accuracy on ImageNet from 87. 6% to 93. 2% and from 88. 0% to 93. 8% on Omniglot compared to competing approaches.
Attend, Infer, Repeat: Fast Scene Understanding with Generative Models
We present a framework for efficient inference in structured image models that explicitly reason about objects.
Exploiting Cyclic Symmetry in Convolutional Neural Networks
We evaluate the effect of these architectural modifications on three datasets which exhibit rotational symmetry and demonstrate improved performance with smaller models.
Asynchronous Methods for Deep Reinforcement Learning
We propose a conceptually simple and lightweight framework for deep reinforcement learning that uses asynchronous gradient descent for optimization of deep neural network controllers.
Ranked #9 on
Atari Games
on Atari 2600 Star Gunner
Pixel Recurrent Neural Networks
Modeling the distribution of natural images is a landmark problem in unsupervised learning.
Ranked #4 on
Image Generation
on Binarized MNIST
Policy Distillation
Policies for complex visual tasks have been successfully learned with deep reinforcement learning, using an approach called deep Q-networks (DQN), but relatively large (task-specific) networks and extensive training are needed to achieve good performance.
Massively Parallel Methods for Deep Reinforcement Learning
We present the first massively distributed architecture for deep reinforcement learning.
Ranked #17 on
Atari Games
on Atari 2600 Private Eye
Natural Neural Networks
We introduce Natural Neural Networks, a novel family of algorithms that speed up convergence by adapting their internal representation during training to improve conditioning of the Fisher matrix.
Spatial Transformer Networks
Convolutional Neural Networks define an exceptionally powerful class of models, but are still limited by the lack of ability to be spatially invariant to the input data in a computationally and parameter efficient manner.
Weight Uncertainty in Neural Networks
We introduce a new, efficient, principled and backpropagation-compatible algorithm for learning a probability distribution on the weights of a neural network, called Bayes by Backprop.
Human level control through deep reinforcement learning
We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester across a set of 49 games, using the same algorithm, network architecture and hyperparameters.
Multiple Object Recognition with Visual Attention
We present an attention-based model for recognizing multiple objects in images.
Recurrent Models of Visual Attention
Applying convolutional neural networks to large images is computationally expensive because the amount of computation scales linearly with the number of image pixels.
Unsupervised Feature Learning by Deep Sparse Coding
In this paper, we propose a new unsupervised feature learning framework, namely Deep Sparse Coding (DeepSC), that extends sparse coding to a multi-layer architecture for visual object recognition tasks.
Playing Atari with Deep Reinforcement Learning
We present the first deep learning model to successfully learn control policies directly from high-dimensional sensory input using reinforcement learning.
Ranked #1 on
Atari Games
on Atari 2600 Pong
Learning word embeddings efficiently with noise-contrastive estimation
Continuous-valued word embeddings learned by neural language models have recently been shown to capture semantic and syntactic information about words very well, setting performance records on several word similarity tasks.
Learning the Dependency Structure of Latent Factors
In this paper, we study latent factor models with the dependency structure in the latent space.
Pedestrian Detection with Unsupervised Multi-Stage Feature Learning
Pedestrian detection is a problem of considerable practical interest.
Natural Language Processing (almost) from Scratch
We propose a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including: part-of-speech tagging, chunking, named entity recognition, and semantic role labeling.
Learning Convolutional Feature Hierarchies for Visual Recognition
We propose an unsupervised method for learning multi-stage hierarchies of sparse convolutional features.
