1 code implementation • 24 Oct 2022 • Kristijan Armeni, Christopher Honey, Tal Linzen
We tested whether language models could retrieve the exact words that occurred previously in a text.
no code implementations • 29 Sep 2021 • Kristijan Armeni, Christopher Honey, Tal Linzen
Thus, although the transformer and LSTM architectures were both trained to predict language sequences, only the transformer learned to flexibly index prior tokens.