no code implementations • 24 May 2024 • Blake Bordelon, Hamza Tahir Chaudhry, Cengiz Pehlevan
In this work, we analyze various scaling limits of the training dynamics of transformer models in the feature learning regime.
1 code implementation • NeurIPS 2023 • Hamza Tahir Chaudhry, Jacob A. Zavatone-Veth, Dmitry Krotov, Cengiz Pehlevan
Sequence memory is an essential attribute of natural and artificial intelligence that enables agents to encode, store, and retrieve complex sequences of stimuli and actions.