Search Results for author: Huashan Sun

Found 5 papers, 1 papers with code

Reviving Dormant Memories: Investigating Catastrophic Forgetting in Language Models through Rationale-Guidance Difficulty

no code implementations18 Nov 2024 Huashan Sun, Yang Gao

In this paper, we discover that when a forgetting model passively receives an externally provided partial appropriate rationale, its performance on the forgotten task can be restored.

Continual Learning

How Far Can In-Context Alignment Go? Exploring the State of In-Context Alignment

no code implementations17 Jun 2024 Heyan Huang, Yinghao Li, Huashan Sun, Yu Bai, Yang Gao

Recent studies have demonstrated that In-Context Learning (ICL), through the use of specific demonstrations, can align Large Language Models (LLMs) with human preferences known as In-Context Alignment (ICA), indicating that models can comprehend human instructions without requiring parameter adjustments.

In-Context Learning Instruction Following

MindLLM: Pre-training Lightweight Large Language Model from Scratch, Evaluations and Domain Applications

no code implementations24 Oct 2023 Yizhe Yang, Huashan Sun, Jiawei Li, Runheng Liu, Yinghao Li, Yuhang Liu, Heyan Huang, Yang Gao

Large Language Models (LLMs) have demonstrated remarkable performance across various natural language tasks, marking significant strides towards general artificial intelligence.

Language Modeling Language Modelling +1

Cannot find the paper you are looking for? You can Submit a new open access paper.