Search Results for author:
Found 13 papers, 6 papers with code
Towards Modular LLMs by Building and Reusing a Library of LoRAs
The growing number of parameter-efficient adaptations of a base large language model (LLM) calls for studying whether we can reuse such trained adapters to improve performance for new tasks.
Guiding Language Model Reasoning with Planning Tokens
To encourage a more structural generation of CoT steps, we propose a hierarchical generation scheme: we let the LM generate a planning token at the start of each reasoning step, intuitively serving as a high-level plan of the current step, and add their embeddings to the model parameters.
Challenging Common Assumptions about Catastrophic Forgetting
In this paper, we study the progressive knowledge accumulation (KA) in DNNs trained with gradient-based algorithms in long sequences of tasks with data re-occurrence.
Continual Learning with Foundation Models: An Empirical Study of Latent Replay
Motivated by this, we study the efficacy of pre-trained vision models as a foundation for downstream continual learning (CL) scenarios.
Continual Learning via Local Module Composition
We introduce local module composition (LMC), an approach to modular CL where each module is provided a local structural component that estimates a module's relevance to the input.
Sequoia: A Software Framework to Unify Continual Learning Research
We propose a taxonomy of settings, where each setting is described as a set of assumptions.
Online Fast Adaptation and Knowledge Accumulation (OSAKA): a New Approach to Continual Learning
The main challenge is that the agent must not forget previous tasks and also adapt to novel tasks in the stream.
Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning
We propose Continual-MAML, an online extension of the popular MAML algorithm as a strong baseline for this scenario.
Pruning at a Glance: Global Neural Pruning for Model Compression
We prove the viability of our method by producing highly compressed models, namely VGG-16, ResNet-56, and ResNet-110 respectively on CIFAR10 without losing any performance compared to the baseline, as well as ResNet-34 and ResNet-50 on ImageNet without a significant loss of accuracy.
Learning to remember: Dynamic Generative Memory for Continual Learning
Continuously trainable models should be able to learn from a stream of data over an undefined period of time.
Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learning
In order to tackle these challenges, we introduce Dynamic Generative Memory (DGM) - a synaptic plasticity driven framework for continual learning.
Ranked #4 on
Continual Learning
on ImageNet-50 (5 tasks)
Self Paced Adversarial Training for Multimodal Few-shot Learning
State-of-the-art deep learning algorithms yield remarkable results in many visual recognition tasks.
Pruning at a Glance: A Structured Class-Blind Pruning Technique for Model Compression
High performance of deep learning models typically comes at cost of considerable model size and computation time.
