Search Results for author:
Found 17 papers, 10 papers with code
Predicting scalar diversity with context-driven uncertainty over alternatives
Here, we test the hypothesis that SI rates depend on the listener’s confidence in the underlying scale, which we operationalize as uncertainty over the distribution of possible alternatives conditioned on the context.
Auxiliary task demands mask the capabilities of smaller language models
Developmental psychologists have argued about when cognitive capacities such as language understanding or theory of mind emerge.
Language models align with human judgments on key grammatical constructions
Do Large Language Models (LLMs) make human-like linguistic generalizations?
Prompting is not a substitute for probability measurements in large language models
Prompting is now a dominant method for evaluating the linguistic knowledge of large language models (LLMs).
Expectations over Unspoken Alternatives Predict Pragmatic Inferences
Here, we test a shared mechanism explaining SI rates within and across scales: context-driven expectations about the unspoken alternatives.
I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragons
We propose a novel task, G4C, to study teacher-student natural language interactions in a goal-driven and grounded environment.
A fine-grained comparison of pragmatic language understanding in humans and language models
We perform a fine-grained comparison of language models and humans on seven pragmatic phenomena, using zero-shot prompting on an expert-curated set of English materials.
Pragmatics in Language Grounding: Phenomena, Tasks, and Modeling Approaches
People rely heavily on context to enrich meaning beyond what is literally said, enabling concise but effective communication.
Controlled Evaluation of Grammatical Knowledge in Mandarin Chinese Language Models
We find suggestive evidence that structural supervision helps with representing syntactic state across intervening content and improves performance in low-data settings, suggesting that the benefits of hierarchical inductive biases in acquiring dependency relationships may extend beyond English.
Scalable pragmatic communication via self-supervision
Models of context-sensitive communication often use the Rational Speech Act framework (RSA; Frank & Goodman, 2012), which formulates listeners and speakers in a cooperative reasoning process.
SyntaxGym: An Online Platform for Targeted Evaluation of Language Models
Targeted syntactic evaluations have yielded insights into the generalizations learned by neural network language models.
On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behavior
Human reading behavior is tuned to the statistics of natural language: the time it takes human subjects to read a word can be predicted from estimates of the word's probability in context.
A Rate-Distortion view of human pragmatic reasoning
What computational principles underlie human pragmatic reasoning?
A Systematic Assessment of Syntactic Generalization in Neural Language Models
While state-of-the-art neural network models continue to achieve lower perplexity scores on language modeling benchmarks, it remains unknown whether optimizing for broad-coverage predictive performance leads to human-like syntactic knowledge.
Generating Bilingual Pragmatic Color References
Contextual influences on language often exhibit substantial cross-lingual regularities; for example, we are more verbose in situations that require finer distinctions.
