Search Results for author:
Found 6 papers, 6 papers with code
Spurious Forgetting in Continual Learning of Language Models
Recent advancements in large language models (LLMs) reveal a perplexing phenomenon in continual learning: despite extensive training, models experience significant performance declines, raising questions about task alignment and underlying knowledge retention.
Towards Lifelong Learning of Large Language Models: A Survey
This survey delves into the sophisticated landscape of lifelong learning, categorizing strategies into two primary groups: Internal Knowledge and External Knowledge.
Incremental Sequence Labeling: A Tale of Two Shifts
As for the E2O problem, we use knowledge distillation to maintain the model's discriminative ability for old entities.
Can LLMs Learn New Concepts Incrementally without Forgetting?
The concepts in Concept-1K are discrete, interpretable units of knowledge that allow for fine-grained analysis of learning and forgetting processes.
Learn or Recall? Revisiting Incremental Learning with Pre-trained Language Models
Most assume that catastrophic forgetting is the biggest obstacle to achieving superior IL performance and propose various techniques to overcome this issue.
Preserving Commonsense Knowledge from Pre-trained Language Models via Causal Inference
Intriguingly, the unified objective can be seen as the sum of the vanilla fine-tuning objective, which learns new knowledge from target data, and the causal objective, which preserves old knowledge from PLMs.
