Exact Learning Dynamics of In-Context Learning in Linear Transformers and Its Application to Non-Linear Transformers

no code implementations • 17 Apr 2025 • Nischal Mainali, Lucas Teixeira

Transformer models exhibit remarkable in-context learning (ICL), adapting to novel tasks from examples within their context, yet the underlying mechanisms remain largely mysterious.

In-Context Learning