How does Multi-Task Training Affect Transformer In-Context Capabilities? Investigations with Function Classes

1 code implementation • 4 Apr 2024 • Harmon Bhasin, Timothy Ossowski, Yiqiao Zhong, Junjie Hu

Large language models (LLM) have recently shown the extraordinary ability to perform unseen tasks based on few-shot examples provided as text, also known as in-context learning (ICL).

In-Context Learning Multi-Task Learning