Search Results for author:
Found 25 papers, 12 papers with code
Large Language Models Pass the Turing Test
We evaluated 4 systems (ELIZA, GPT-4o, LLaMa-3. 1-405B, and GPT-4. 5) in two randomised, controlled, and pre-registered Turing tests on independent populations.
Lies, Damned Lies, and Distributional Language Statistics: Persuasion and Deception with Large Language Models
Large Language Models (LLMs) can generate content that is as persuasive as human-written text and appear capable of selectively producing deceptive outputs.
Why do language models perform worse for morphologically complex languages?
We then propose and test three possible causes for this performance gap: morphological alignment of tokenizers, tokenization quality, and disparities in dataset sizes and measurement.
Goldfish: Monolingual Language Models for 350 Languages
For many low-resource languages, the only available language models are large multilingual models trained on many languages simultaneously.
GPT-4 is judged more human than humans in displaced and inverted Turing tests
GPT-3. 5, GPT-4, and displaced human adjudicators judged whether an agent was human or AI on the basis of a Turing test transcript.
Dissecting the Ullman Variations with a SCALPEL: Why do LLMs fail at Trivial Alterations to the False Belief Task?
Recent empirical results have sparked a debate about whether or not Large Language Models (LLMs) are capable of Theory of Mind (ToM).
People cannot distinguish GPT-4 from a human in a Turing test
We evaluated 3 systems (ELIZA, GPT-3. 5 and GPT-4) in a randomized, controlled, and preregistered Turing test.
Revenge of the Fallen? Recurrent Models Match Transformers at Predicting Human Language Comprehension Metrics
Transformers have generally supplanted recurrent neural networks as the dominant architecture for both natural language processing tasks and for modelling the effect of predictability on online human language comprehension.
A Bit of a Problem: Measurement Disparities in Dataset Sizes Across Languages
We release a tool to obtain byte premiums for any two languages, enabling comparisons of dataset sizes across languages for more equitable multilingual model development and data practices.
When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages
However, concrete evidence for the effects of multilinguality on language modeling performance in individual languages remains scarce.
Structural Priming Demonstrates Abstract Grammatical Representations in Multilingual Language Models
We measure crosslingual structural priming in large language models, comparing model behavior to human experimental results from eight crosslingual experiments covering six languages, and four monolingual structural priming experiments in three non-English languages.
Crosslingual Structural Priming and the Pre-Training Dynamics of Bilingual Language Models
Do multilingual language models share abstract grammatical representations across languages, and if so, when do these develop?
Characterizing Learning Curves During Language Model Pre-Training: Learning, Forgetting, and Stability
To better understand these fluctuations, we quantify the final surprisal, within-run variability, age of acquisition, forgettability, and cross-run variability of learning curves for individual tokens in context.
Emergent inabilities? Inverse scaling over the course of pretraining
Does inverse scaling only occur as a function of model parameter size, or can it also occur over the course of training?
Language Model Behavior: A Comprehensive Survey
Transformer language models have received widespread public attention, yet their generated text is often surprising even to NLP researchers.
Can Peanuts Fall in Love with Distributional Semantics?
Context changes expectations about upcoming words - following a story involving an anthropomorphic peanut, comprehenders expect the sentence the peanut was in love more than the peanut was salted, as indexed by N400 amplitude (Nieuwland & van Berkum, 2006).
Rarely a problem? Language models exhibit inverse scaling in their predictions following few-type quantifiers
How well do language models deal with quantification?
Collateral facilitation in humans and language models
Are the predictions of humans and language models affected by similar things?
Do language models make human-like predictions about the coreferents of Italian anaphoric zero pronouns?
Some languages allow arguments to be omitted in certain contexts.
The Geometry of Multilingual Language Model Representations
The subspace means differ along language-sensitive axes that are relatively stable throughout middle layers, and these axes encode information such as token vocabularies.
Word Acquisition in Neural Language Models
We investigate how neural language models acquire individual words during training, extracting learning curves and ages of acquisition for over 600 words on the MacArthur-Bates Communicative Development Inventory (Fenson et al., 2007).
So Cloze yet so Far: N400 Amplitude is Better Predicted by Distributional Information than Human Predictability Judgements
In this study, we investigate whether the linguistic predictions of computational language models or humans better reflect the way in which natural language stimuli modulate the amplitude of the N400.
Different kinds of cognitive plausibility: why are transformers better than RNNs at predicting N400 amplitude?
Despite being designed for performance rather than cognitive plausibility, transformer language models have been found to be better at predicting metrics used to assess human language comprehension than language models with other architectures, such as recurrent neural networks.
How well does surprisal explain N400 amplitude under different experimental conditions?
We investigate the extent to which word surprisal can be used to predict a neural measure of human language processing difficulty - the N400.
