no code implementations • 30 Apr 2024 • James A. Michaelov, Catherine Arnett, Benjamin K. Bergen
Transformers have supplanted Recurrent Neural Networks as the dominant architecture for both natural language processing tasks and, despite criticisms of cognitive implausibility, for modelling the effect of predictability on online human language comprehension.
no code implementations • 20 Mar 2024 • Catherine Arnett, Pamela D. Rivière, Tyler A. Chang, Sean Trott
The relationship between language model tokenization and performance is an open area of research.
1 code implementation • 1 Mar 2024 • Catherine Arnett, Tyler A. Chang, Benjamin K. Bergen
We release a tool to obtain byte premiums for any two languages, enabling comparisons of dataset sizes across languages for more equitable multilingual model development and data practices.
1 code implementation • 15 Nov 2023 • Tyler A. Chang, Catherine Arnett, Zhuowen Tu, Benjamin K. Bergen
However, concrete evidence for the effects of multilinguality on language modeling performance in individual languages remains scarce.
no code implementations • 15 Nov 2023 • James A. Michaelov, Catherine Arnett, Tyler A. Chang, Benjamin K. Bergen
We measure crosslingual structural priming in large language models, comparing model behavior to human experimental results from eight crosslingual experiments covering six languages, and four monolingual structural priming experiments in three non-English languages.
no code implementations • 11 Oct 2023 • Catherine Arnett, Tyler A. Chang, James A. Michaelov, Benjamin K. Bergen
Do multilingual language models share abstract grammatical representations across languages, and if so, when do these develop?