In-context Learning and Gradient Descent Revisited

1 code implementation • 13 Nov 2023 • Gilad Deutch, Nadav Magar, Tomer Bar Natan, Guy Dar

Next, we explore a major discrepancy in the flow of information throughout the model between ICL and GD, which we term Layer Causality.

Few-Shot Learning In-Context Learning