Search Results for author:
Found 19 papers, 13 papers with code
On the Computation of the Fisher Information in Continual Learning
One of the most popular methods for continual learning with deep neural networks is Elastic Weight Consolidation (EWC), which involves computing the Fisher Information.
Aligning Generalisation Between Humans and Machines
A crucial yet often overlooked aspect of these interactions is the different ways in which humans and machines generalise.
Lifelong Learning of Video Diffusion Models From a Single Video Stream
This work demonstrates that training autoregressive video diffusion models from a single, continuous video stream is not only possible but remarkably can also be competitive with standard offline training approaches given the same number of gradient steps.
Continual Learning in the Presence of Repetition
Continual learning (CL) provides a framework for training models in ever-evolving environments.
Continual Learning and Catastrophic Forgetting
This book chapter delves into the dynamics of continual learning, which is the process of incrementally learning from a non-stationary stream of data.
Infinite dSprites for Disentangled Continual Learning: Separating Memory Edits from Generalization
In a simple setting with direct supervision on the generative factors, we show how learning class-agnostic transformations offers a way to circumvent catastrophic forgetting and improve classification accuracy over time.
Continual Learning of Diffusion Models with Generative Distillation
Diffusion models are powerful generative models that achieve state-of-the-art performance in image synthesis.
Continual Learning: Applications and the Road Forward
Continual learning is a subfield of machine learning, which aims to allow machine learning models to continuously learn on new data, by accumulating knowledge without forgetting what was learned in the past.
Two Complementary Perspectives to Continual Learning: Ask Not Only What to Optimize, But Also How
While there is some continual learning work that alters the optimization trajectory (e. g., using gradient projection techniques), this line of research is positioned as alternative to improving the optimization objective, while we argue it should be complementary.
Prediction Error-based Classification for Class-Incremental Learning
Class-incremental learning (CIL) is a particularly challenging variant of continual learning, where the goal is to learn to discriminate between all classes presented in an incremental fashion.
Knowledge Accumulation in Continually Learned Representations and the Issue of Feature Forgetting
Continual learning research has shown that neural networks suffer from catastrophic forgetting "at the output level", but it is debated whether this is also the case at the level of learned representations.
Natural continual learning: success is a journey, not (just) a destination
In contrast, artificial agents are prone to 'catastrophic forgetting' whereby performance on previous tasks deteriorates rapidly as new ones are acquired.
Class-Incremental Learning with Generative Classifiers
As a proof-of-principle, here we implement this strategy by training a variational autoencoder for each class to be learned and by using importance sampling to estimate the likelihoods p(x|y).
Energy-Based Models for Continual Learning
We motivate Energy-Based Models (EBMs) as a promising model class for continual learning problems.
Brain-inspired replay for continual learning with artificial neural networks
In artificial neural networks, such memory replay can be implemented as ‘generative replay’, which can successfully – and surprisingly efficiently – prevent catastrophic forgetting on toy examples even in a class-incremental learning scenario.
Omnidirectional Transfer for Quasilinear Lifelong Learning
But striving to avoid forgetting sets the goal unnecessarily low: the goal of lifelong learning, whether biological or artificial, should be to improve performance on all tasks (including past and future) with any new data.
Three scenarios for continual learning
Standard artificial neural networks suffer from the well-known issue of catastrophic forgetting, making continual or lifelong learning difficult for machine learning.
Generative replay with feedback connections as a general strategy for continual learning
A major obstacle to developing artificial intelligence applications capable of true lifelong learning is that artificial neural networks quickly or catastrophically forget previously learned tasks when trained on a new one.
Three continual learning scenarios and a case for generative replay
To enable more meaningful comparisons, we identified three distinct continual learning scenarios based on whether task identity is known and, if it is not, whether it needs to be inferred.
