Search Results for author:
Found 102 papers, 45 papers with code
Uncovering mesa-optimization algorithms in Transformers
Transformers have become the dominant model in deep learning, but the reason for their superior performance is poorly understood.
On the Universality of Linear Recurrences Followed by Nonlinear Projections
In this note (work in progress towards a full-length paper) we show that a family of sequence models based on recurrent linear layers~(including S4, S5, and the LRU) interleaved with position-wise multi-layer perceptrons~(MLPs) can approximate arbitrarily well any sufficiently regular non-linear sequence-to-sequence map.
Promoting Exploration in Memory-Augmented Adam using Critical Momenta
Adaptive gradient-based optimizers, particularly Adam, have left their mark in training large-scale deep learning models.
Latent Space Representations of Neural Algorithmic Reasoners
Neural Algorithmic Reasoning (NAR) is a research area focused on designing neural architectures that can reliably capture classical computation, usually by learning to execute algorithms.
Towards Robust and Efficient Continual Language Learning
As the application space of language models continues to evolve, a natural question to ask is how we can quickly adapt models to new tasks.
Asynchronous Algorithmic Alignment with Cocycles
State-of-the-art neural algorithmic reasoners make use of message passing in graph neural networks (GNNs).
Learning to Modulate pre-trained Models in RL
However, fine-tuning a pre-trained model often suffers from catastrophic forgetting, that is, the performance on the pre-training tasks deteriorates when fine-tuning on new tasks.
Kalman Filter for Online Classification of Non-Stationary Data
Non-stationarity over the linear predictor weights is modelled using a parameter drift transition density, parametrized by a coefficient that quantifies forgetting.
The Tunnel Effect: Building Data Representations in Deep Neural Networks
Deep neural networks are widely known for their remarkable effectiveness across various tasks, with the consensus that deeper networks implicitly learn more complex data representations.
Deep Reinforcement Learning with Plasticity Injection
A growing body of evidence suggests that neural networks employed in deep reinforcement learning (RL) gradually lose their plasticity, the ability to learn from new data; however, the analysis and mitigation of this phenomenon is hampered by the complex relationship between plasticity, exploration, and performance in RL.
Towards Compute-Optimal Transfer Learning
The field of transfer learning is undergoing a significant shift with the introduction of large pretrained models which have demonstrated strong adaptability to a variety of downstream tasks.
Resurrecting Recurrent Neural Networks for Long Sequences
Recurrent Neural Networks (RNNs) offer fast inference on long sequences but are hard to optimize and slow to train.
Understanding plasticity in neural networks
Plasticity, the ability of a neural network to quickly change its predictions in response to new information, is essential for the adaptability and robustness of deep reinforcement learning systems.
SemPPL: Predicting pseudo-labels for better contrastive representations
We propose a new semi-supervised learning method, Semantic Positives via Pseudo-Labels (SemPPL), that combines labelled and unlabelled data to learn informative representations.
NEVIS'22: A Stream of 100 Tasks Sampled from 30 Years of Computer Vision Research
A shared goal of several machine learning communities like continual learning, meta-learning and transfer learning, is to design algorithms and models that efficiently and robustly adapt to unseen tasks.
Probing Transfer in Deep Reinforcement Learning without Task Engineering
By formally organising these modifications into several factors of variation, we are able to show that Analyses of Variance (ANOVA) are a potent tool for studying the effects of human-relevant domain changes on the learning and transfer performance of a deep reinforcement learning agent.
Disentangling Transfer in Continual Reinforcement Learning
The ability of continual learning systems to transfer knowledge from previously seen tasks in order to maximize performance on new tasks is a significant challenge for the field, limiting the applicability of continual learning solutions to realistic scenarios.
An Empirical Study of Implicit Regularization in Deep Offline RL
Also, we empirically identify three phases of learning that explain the impact of implicit regularization on the learning dynamics and found that bootstrapping alone is insufficient to explain the collapse of the effective rank.
When Does Re-initialization Work?
However, when deployed alongside other carefully tuned regularization techniques, re-initialization methods offer little to no added benefit for generalization, although optimal generalization performance becomes less sensitive to the choice of learning rate and weight decay hyperparameters.
Pre-training via Denoising for Molecular Property Prediction
Many important problems involving molecular property prediction from 3D structures have limited data, posing a generalization challenge for neural networks.
The CLRS Algorithmic Reasoning Benchmark
Learning representations of algorithms is an emerging area of machine learning, seeking to bridge concepts from neural networks with classical algorithms.
Architecture Matters in Continual Learning
However, in this work, we show that the choice of architecture can significantly impact the continual learning performance, and different architectures lead to different trade-offs between the ability to remember previous tasks and learning new ones.
Pushing the limits of self-supervised ResNets: Can we outperform supervised learning without labels on ImageNet?
Most notably, ReLICv2 is the first unsupervised representation learning method to consistently outperform the supervised baseline in a like-for-like comparison over a range of ResNet architectures.
Ranked #13 on
Semantic Segmentation
on PASCAL VOC 2012 val
Representation Learning
Self-Supervised Image Classification
+3
Wide Neural Networks Forget Less Catastrophically
A primary focus area in continual learning research is alleviating the "catastrophic forgetting" problem in neural networks by designing new algorithms that are more robust to the distribution shifts.
Powerpropagation: A sparsity inducing weight reparameterisation
The training of sparse neural networks is becoming an increasingly important tool for reducing the computational footprint of models at training and evaluation, as well enabling the effective scaling up of models.
On the Role of Optimization in Double Descent: A Least Squares Study
Empirically it has been observed that the performance of deep neural networks steadily improves as we increase model size, contradicting the classical view on overfitting and generalization.
Reasoning-Modulated Representations
Neural networks leverage robust internal representations in order to generalise.
Task-agnostic Continual Learning with Hybrid Probabilistic Models
Learning new tasks continuously without forgetting on a constantly changing data distribution is essential for real-world problems but extremely challenging for modern deep learning.
Test Sample Accuracy Scales with Training Sample Density in Neural Networks
Intuitively, one would expect accuracy of a trained neural network's prediction on test samples to correlate with how densely the samples are surrounded by seen training samples in representation space.
Top-KAST: Top-K Always Sparse Training
Sparse neural networks are becoming increasingly important as the field seeks to improve the performance of existing models by scaling them up, while simultaneously trying to reduce power consumption and computational footprint.
A study on the plasticity of neural networks
One aim shared by multiple settings, such as continual learning or transfer learning, is to leverage previously acquired knowledge to converge faster on the current task.
Drawing Multiple Augmentation Samples Per Image During Training Efficiently Decreases Test Error
In this work, we provide a detailed empirical evaluation of how the number of augmentation samples per unique image influences model performance on held out data when training deep ResNets.
Ranked #116 on
Image Classification
on ImageNet
Continual World: A Robotic Benchmark For Continual Reinforcement Learning
Continual learning (CL) -- the ability to continuously learn, building on previously acquired knowledge -- is a natural requirement for long-lived autonomous reinforcement learning (RL) agents.
Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspective
We conduct ablation studies to disentangle the various effects normalisation has on the learning dynamics and show that is sufficient to modulate the parameter updates to recover most of the performance of spectral normalisation.
Regularized Behavior Value Estimation
Due to bootstrapping, these errors get amplified during training and can lead to divergence, thereby crippling learning.
Addressing Extrapolation Error in Deep Offline Reinforcement Learning
These errors can be compounded by bootstrapping when the function approximator overestimates, leading the value function to *grow unbounded*, thereby crippling learning.
Behavior Priors for Efficient Reinforcement Learning
In this work we consider how information and architectural constraints can be combined with ideas from the probabilistic modeling literature to learn behavior priors that capture the common movement and interaction patterns that are shared across a set of related tasks or contexts.
BYOL works even without batch statistics
Bootstrap Your Own Latent (BYOL) is a self-supervised learning approach for image representation.
Linear Mode Connectivity in Multitask and Continual Learning
Continual (sequential) training and multitask (simultaneous) training are often attempting to solve the same overall objective: to find a solution that performs well on all considered tasks.
Temporal Difference Uncertainties as a Signal for Exploration
Instead, we incorporate it as an intrinsic reward and treat exploration as a separate learning problem, induced by the agent's temporal difference uncertainties.
Understanding the Role of Training Regimes in Continual Learning
However, there has been limited prior work extensively analyzing the impact that different training regimes -- learning rate, batch size, regularization method-- can have on forgetting.
Pointer Graph Networks
This static input structure is often informed purely by insight of the machine learning practitioner, and might not be optimal for the actual task the GNN is solving.
Multiplicative Interactions and Where to Find Them
We explore the role of multiplicative interaction as a unifying framework to describe a range of classical and modern neural network architectural motifs, such as gating, attention layers, hypernetworks, and dynamic convolutions amongst others.
A Deep Neural Network's Loss Surface Contains Every Low-dimensional Pattern
The work "Loss Landscape Sightseeing with Multi-Point Optimization" (Skorokhodov and Burtsev, 2019) demonstrated that one can empirically find arbitrary 2D binary patterns inside loss surfaces of popular neural networks.
Continual Unsupervised Representation Learning
Continual learning aims to improve the ability of modern learning systems to deal with non-stationary distributions, typically by attempting to learn a series of tasks sequentially.
Improving the Gating Mechanism of Recurrent Neural Networks
Gating mechanisms are widely used in neural network models, where they allow gradients to backpropagate more easily through depth or time.
Stabilizing Transformers for Reinforcement Learning
Harnessing the transformer's ability to process long time horizons of information could provide a similar performance boost in partially observable reinforcement learning (RL) domains, but the large-scale transformers used in NLP have yet to be successfully applied to the RL setting.
Meta-Learning with Warped Gradient Descent
On the other hand, approaches that try to control a gradient-based update rule typically resort to computing gradients through the learning process to obtain their meta-gradients, leading to methods that can not scale beyond few-shot task adaptation.
Task Agnostic Continual Learning via Meta Learning
One particular formalism that studies learning under non-stationary distribution is provided by continual learning, where the non-stationarity is imposed by a sequence of distinct tasks.
Meta-learning of Sequential Strategies
In this report we review memory-based meta-learning as a tool for building sample-efficient strategies that learn from past experience to adapt to any task within a target class.
Information asymmetry in KL-regularized RL
In this work we study the possibility of leveraging such repeated structure to speed up and regularize learning.
Deep reinforcement learning with relational inductive biases
We introduce an approach for augmenting model-free deep reinforcement learning agents with a mechanism for relational reasoning over structured representations, which improves performance, learning efficiency, generalization, and interpretability.
Ray Interference: a Source of Plateaus in Deep Reinforcement Learning
Rather than proposing a new method, this paper investigates an issue present in existing learning algorithms.
A RAD approach to deep mixture models
Flow based models such as Real NVP are an extremely powerful approach to density estimation.
Exploiting Hierarchy for Learning and Transfer in KL-regularized RL
As reinforcement learning agents are tasked with solving more challenging and diverse tasks, the ability to incorporate prior knowledge into the learning system and to exploit reusable structure in solution space is likely to become increasingly important.
Distilling Policy Distillation
The transfer of knowledge from one policy to another is an important tool in Deep Reinforcement Learning.
Functional Regularisation for Continual Learning with Gaussian Processes
We introduce a framework for Continual Learning (CL) based on Bayesian inference over the function space rather than the parameters of a deep neural network.
Adapting Auxiliary Losses Using Gradient Similarity
One approach to deal with the statistical inefficiency of neural networks is to rely on auxiliary losses that help to build useful representations.
Meta-Learning with Latent Embedding Optimization
We show that it is possible to bypass these limitations by learning a data-dependent latent generative representation of model parameters, and performing gradient-based meta-learning in this low-dimensional latent space.
Mix & Match - Agent Curricula for Reinforcement Learning
We introduce Mix and match (M&M) – a training framework designed to facilitate rapid and effective learning in RL agents that would be too slow or too challenging to train otherwise. The key innovation is a procedure that allows us to automatically form a curriculum over agents.
Relational Deep Reinforcement Learning
We introduce an approach for deep reinforcement learning (RL) that improves upon the efficiency, generalization capacity, and interpretability of conventional approaches through structured perception and relational reasoning.
Relational recurrent neural networks
Memory-based neural networks model temporal data by leveraging an ability to remember information for long periods.
Ranked #59 on
Language Modelling
on WikiText-103
Mix&Match - Agent Curricula for Reinforcement Learning
(2) We further show that M&M can be used successfully to progress through a curriculum of architectural variants defining an agents internal state.
Relational inductive biases, deep learning, and graph networks
As a companion to this paper, we have released an open-source software library for building graph networks, with demonstrations of how to use them in practice.
Been There, Done That: Meta-Learning with Episodic Recall
Meta-learning agents excel at rapidly learning new tasks from open-ended task distributions; yet, they forget what they learn about each task as soon as the next begins.
Hyperbolic Attention Networks
We introduce hyperbolic attention networks to endow neural networks with enough capacity to match the complexity of data with hierarchical and power-law structure.
Progress & Compress: A scalable framework for continual learning
This is achieved by training a network with two components: A knowledge base, capable of solving previously encountered problems, which is connected to an active column that is employed to efficiently learn the current task.
Low-pass Recurrent Neural Networks - A memory architecture for longer-term correlation discovery
Reinforcement learning (RL) agents performing complex tasks must be able to remember observations and actions across sizable time intervals.
Block Mean Approximation for Efficient Second Order Optimization
Advanced optimization algorithms such as Newton method and AdaGrad benefit from second order derivative or second order statistics to achieve better descent directions and faster convergence rates.
Learning Deep Generative Models of Graphs
Graphs are fundamental data structures which concisely capture the relational structure in many important real-world domains, such as knowledge graphs, physical and social interactions, language, and chemistry.
Memory-based Parameter Adaptation
Deep neural networks have excelled on a wide range of problems, from vision to language and game playing.
Model compression via distillation and quantization
Deep neural networks (DNNs) continue to make significant advances, solving tasks from image classification to translation or reinforcement learning.
Visual Interaction Networks: Learning a Physics Simulator from Video
We introduce the Visual Interaction Network, a general-purpose model for learning the dynamics of a physical system from raw visual observations.
Imagination-Augmented Agents for Deep Reinforcement Learning
We introduce Imagination-Augmented Agents (I2As), a novel architecture for deep reinforcement learning combining model-free and model-based aspects.
Model-based Reinforcement Learning
reinforcement-learning
+1
Learning model-based planning from scratch
Here we introduce the "Imagination-based Planner", the first model-based, sequential decision-making agent that can learn to construct, evaluate, and execute plans.
Distral: Robust Multitask Reinforcement Learning
Moreover, the proposed learning process is more robust and more stable---attributes that are critical in deep reinforcement learning.
Sobolev Training for Neural Networks
In many cases we only have access to input-output pairs from the ground truth, however it is becoming more common to have access to derivatives of the target output with respect to the input - for example when the ground truth function is itself a neural network such as in network compression or distillation.
Visual Interaction Networks
We found that from just six input video frames the Visual Interaction Network can generate accurate future trajectories of hundreds of time steps on a wide range of physical systems.
A simple neural network module for relational reasoning
Relational reasoning is a central component of generally intelligent behavior, but has proven difficult for neural networks to learn.
Image Retrieval with Multi-Modal Query
Question Answering
+2
Metacontrol for Adaptive Imagination-Based Optimization
The metacontroller component is a model-free reinforcement learning agent, which decides both how many iterations of the optimization procedure to run, as well as which model to consult on each iteration.
Sharp Minima Can Generalize For Deep Nets
Despite their overwhelming capacity to overfit, deep learning architectures tend to generalize relatively well to unseen data, allowing them to be deployed in practice.
Discovering objects and their relations from entangled scene representations
We show that RNs are capable of learning object relations from scene description data.
Overcoming catastrophic forgetting in neural networks
The ability to learn tasks in a sequential fashion is crucial to the development of artificial intelligence.
Ranked #3 on
Continual Learning
on F-CelebA (10 tasks)
Interaction Networks for Learning about Objects, Relations and Physics
Here we introduce the interaction network, a model which can reason about how objects in complex systems interact, supporting dynamical predictions, as well as inferences about the abstract properties of the system.
Local minima in training of neural networks
Given that deep networks are highly nonlinear systems optimized by local gradient methods, why do they not seem to be affected by bad local minima?
Learning to Navigate in Complex Environments
Learning to navigate in complex environments with dynamic elements is an important milestone in developing AI agents.
Sim-to-Real Robot Learning from Pixels with Progressive Nets
The progressive net approach is a general framework that enables reuse of everything from low-level visual features to high-level policies for transfer to new tasks, enabling a compositional, yet simple, approach to building complex skills.
Progressive Neural Networks
Learning to solve complex sequences of tasks--while both leveraging transfer and avoiding catastrophic forgetting--remains a key obstacle to achieving human-level intelligence.
Theano: A Python framework for fast computation of mathematical expressions
Since its introduction, it has been one of the most used CPU and GPU mathematical compilers - especially in the machine learning community - and has shown steady performance improvements.
Policy Distillation
Policies for complex visual tasks have been successfully learned with deep reinforcement learning, using an approach called deep Q-networks (DQN), but relatively large (task-specific) networks and extensive training are needed to achieve good performance.
Natural Neural Networks
We introduce Natural Neural Networks, a novel family of algorithms that speed up convergence by adapting their internal representation during training to improve conditioning of the Fisher matrix.
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
Gradient descent or quasi-Newton methods are almost ubiquitously used to perform such minimizations, and it is often thought that a main source of difficulty for these local methods to find the global minimum is the proliferation of local minima with much higher error than the global minimum.
On the saddle point problem for non-convex optimization
Gradient descent or quasi-Newton methods are almost ubiquitously used to perform such minimizations, and it is often thought that a main source of difficulty for the ability of these local methods to find the global minimum is the proliferation of local minima with much higher error than the global minimum.
On the Number of Linear Regions of Deep Neural Networks
We study the complexity of functions computable by deep feedforward neural networks with piecewise linear activations in terms of the symmetries and the number of linear regions that they have.
How to Construct Deep Recurrent Neural Networks
Based on this observation, we propose two novel architectures of a deep RNN which are orthogonal to an earlier attempt of stacking multiple recurrent layers to build a deep RNN (Schmidhuber, 1992; El Hihi and Bengio, 1996).
On the number of response regions of deep feed forward networks with piece-wise linear activations
For a $k$ layer model with $n$ hidden units on each layer it is $\Omega(\left\lfloor {n}/{n_0}\right\rfloor^{k-1}n^{n_0})$.
Learned-Norm Pooling for Deep Feedforward and Recurrent Neural Networks
In this paper we propose and investigate a novel nonlinear unit, called $L_p$ unit, for deep neural networks.
Revisiting Natural Gradient for Deep Networks
We evaluate natural gradient, an algorithm originally proposed in Amari (1997), for learning deep models.
Advances in Optimizing Recurrent Networks
After a more than decade-long period of relatively little research activity in the area of recurrent neural networks, several new developments will be reviewed here that have allowed substantial progress both in understanding and in technical solutions towards more efficient training of recurrent networks.
Theano: new features and speed improvements
Theano is a linear algebra compiler that optimizes a user's symbolically-specified mathematical computations to produce efficient low-level implementations.
On the difficulty of training Recurrent Neural Networks
There are two widely known issues with properly training Recurrent Neural Networks, the vanishing and the exploding gradient problems detailed in Bengio et al. (1994).
