Search Results for author:
Found 10 papers, 6 papers with code
Parameter-Level Soft-Masking for Continual Learning
Although several techniques have achieved learning with no CF, they attain it by letting each task monopolize a sub-network in a shared network, which seriously limits knowledge transfer (KT) and causes over-consumption of the network capacity, i. e., as more tasks are learned, the performance deteriorates.
Open-World Continual Learning: Unifying Novelty Detection and Continual Learning
The key theoretical result is that regardless of whether WP and OOD detection (or TP) are defined explicitly or implicitly by a CIL algorithm, good WP and good OOD detection are necessary and sufficient conditions for good CIL, which unifies novelty or OOD detection and continual learning (CIL, in particular).
Continual Pre-training of Language Models
A novel proxy is also proposed to preserve the general knowledge in the original LM.
Ranked #1 on
Continual Pretraining
on ACL-ARC
A Theoretical Study on Solving Continual Learning
Continual learning (CL) learns a sequence of tasks incrementally.
A Multi-Head Model for Continual Learning via Out-of-Distribution Replay
Instead of using the saved samples in memory to update the network for previous tasks/classes in the existing approach, MORE leverages the saved samples to build a task specific classifier (adding a new classification head) without updating the network learned for previous tasks/classes.
Continual Learning Based on OOD Detection and Task Masking
Existing continual learning techniques focus on either task incremental learning (TIL) or class incremental learning (CIL) problem, but not both.
Partially Relaxed Masks for Lightweight Knowledge Transfer without Forgetting in Continual Learning
The goal of this work is to endow such systems with the additional ability to transfer knowledge among tasks when the tasks are similar and have shared knowledge to achieve higher accuracy.
Continual Learning Using Pseudo-Replay via Latent Space Sampling
PLS is not only simple and efficient but also does not invade data privacy due to the fact that it works in the latent feature space.
Continual Learning via Principal Components Projection
The idea is that in learning a new task, if we can ensure that the gradient updates will only occur in the orthogonal directions to the input vectors of the previous tasks, then the weight updates for learning the new task will not affect the previous tasks.
