Search Results for author:
Found 15 papers, 7 papers with code
Automated Curriculum Learning for Neural Networks
We introduce a method for automatically selecting the path, or syllabus, that a neural network follows through a curriculum so as to maximise learning efficiency.
Noisy Networks for Exploration
We introduce NoisyNet, a deep reinforcement learning agent with parametric noise added to its weights, and show that the induced stochasticity of the agent's policy can be used to aid efficient exploration.
Ranked #1 on
Atari Games
on Atari 2600 Surround
Associative Compression Networks for Representation Learning
We conclude that ACNs are a promising new direction for representation learning: one that steps away from IID modelling, and towards learning a structured description of the dataset as a whole.
Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscaling
To address the latter challenge, we propose to use Multidimensional Upscaling to grow an image in both size and depth via intermediate stages utilising distinct SPNs.
Ranked #2 on
Image Generation
on CelebA 256x256
Rigging the Lottery: Making All Tickets Winners
There is a large body of work on training dense networks to yield sparse networks for inference, but this limits the size of the largest trainable sparse model to that of the largest trainable dense model.
Ranked #1 on
Sparse Learning
on ImageNet
Multiplicative Interactions and Where to Find Them
We explore the role of multiplicative interaction as a unifying framework to describe a range of classical and modern neural network architectural motifs, such as gating, attention layers, hypernetworks, and dynamic convolutions amongst others.
A Practical Sparse Approximation for Real Time Recurrent Learning
Current methods for training recurrent neural networks are based on backpropagation through time, which requires storing a complete history of network states, and prohibits updating the weights `online' (after every timestep).
Practical Real Time Recurrent Learning with a Sparse Approximation
For highly sparse networks, SnAp with $n=2$ remains tractable and can outperform backpropagation through time in terms of learning speed when updates are done online.
Generating Images with Sparse Representations
The high dimensionality of images presents architecture and sampling-efficiency challenges for likelihood-based generative models.
Multimodal Few-Shot Learning with Frozen Language Models
When trained at sufficient scale, auto-regressive language models exhibit the notable ability to learn a new language task after being prompted with just a few examples.
Ranked #11 on
Visual Question Answering (VQA)
on VQA v2 val
Improving language models by retrieving from trillions of tokens
We enhance auto-regressive language models by conditioning on document chunks retrieved from a large corpus, based on local similarity with preceding tokens.
Ranked #1 on
Language Modelling
on WikiText-103
(using extra training data)
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.
Ranked #1 on
Language Modelling
on StackExchange
Teaching language models to support answers with verified quotes
We measure the performance of GopherCite by conducting human evaluation of answers to questions in a subset of the NaturalQuestions and ELI5 datasets.
Flamingo: a Visual Language Model for Few-Shot Learning
Building models that can be rapidly adapted to novel tasks using only a handful of annotated examples is an open challenge for multimodal machine learning research.
Ranked #1 on
Action Recognition
on RareAct
GPT-4 Technical Report
We report the development of GPT-4, a large-scale, multimodal model which can accept image and text inputs and produce text outputs.
Ranked #1 on
Long-Context Understanding
on Ada-LEval (BestAnswer)
