Search Results for author:
Found 11 papers, 7 papers with code
Preserving Knowledge in Large Language Model with Model-Agnostic Self-Decompression
Humans can retain old knowledge while learning new information, but Large Language Models (LLMs) often suffer from catastrophic forgetting when post-pretrained or supervised fine-tuned (SFT) on domain-specific data.
HORAE: A Domain-Agnostic Modeling Language for Automating Multimodal Service Regulation
Artificial intelligence is rapidly encroaching on the field of service regulation.
You Only Cache Once: Decoder-Decoder Architectures for Language Models
We introduce a decoder-decoder architecture, YOCO, for large language models, which only caches key-value pairs once.
Retentive Network: A Successor to Transformer for Large Language Models
In this work, we propose Retentive Network (RetNet) as a foundation architecture for large language models, simultaneously achieving training parallelism, low-cost inference, and good performance.
Debiased Inference for Dynamic Nonlinear Panels with Multi-dimensional Heterogeneities
We introduce a generic class of dynamic nonlinear heterogeneous parameter models that incorporate individual and time effects in both the intercept and slope.
Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers
We comprehensively compare the behaviors of in-context learning and explicit finetuning on real tasks to provide empirical evidence that supports our understanding.
Structured Prompting: Scaling In-Context Learning to 1,000 Examples
Large language models have exhibited intriguing in-context learning capability, achieving promising zero- and few-shot performance without updating the parameters.
Prototypical Calibration for Few-shot Learning of Language Models
In-context learning of GPT-like models has been recognized as fragile across different hand-crafted templates, and demonstration permutations.
