Search Results for author:
Found 59 papers, 23 papers with code
Flexible Generation from Fragmentary Linguistic Input
We hypothesize that human performance is better characterized by flexible inference through composition of basic computational motifs available to the human language user.
Predicting scalar diversity with context-driven uncertainty over alternatives
Here, we test the hypothesis that SI rates depend on the listener’s confidence in the underlying scale, which we operationalize as uncertainty over the distribution of possible alternatives conditioned on the context.
Controlled Evaluation of Syntactic Knowledge in Multilingual Language Models
Language models (LMs) are capable of acquiring elements of human-like syntactic knowledge.
Elements of World Knowledge (EWOK): A cognition-inspired framework for evaluating basic world knowledge in language models
We present Elements of World Knowledge (EWOK), a framework for evaluating world modeling in language models by testing their ability to use knowledge of a concept to match a target text with a plausible/implausible context.
Learning Phonotactics from Linguistic Informants
We propose an interactive approach to language learning that utilizes linguistic acceptability judgments from an informant (a competent language user) to learn a grammar.
Language models align with human judgments on key grammatical constructions
Do large language models (LLMs) make human-like linguistic generalizations?
LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers
Logical reasoning, i. e., deductively inferring the truth value of a conclusion from a set of premises, is an important task for artificial intelligence with wide potential impacts on science, mathematics, and society.
Probing self-supervised speech models for phonetic and phonemic information: a case study in aspiration
Textless self-supervised speech models have grown in capabilities in recent years, but the nature of the linguistic information they encode has not yet been thoroughly examined.
A Cross-Linguistic Pressure for Uniform Information Density in Word Order
Here, we ask whether a pressure for UID may have influenced word order patterns cross-linguistically.
The neural dynamics of auditory word recognition and integration
We fit this model to explain scalp EEG signals recorded as subjects passively listened to a fictional story, revealing both the dynamics of the online auditory word recognition process and the neural correlates of the recognition and integration of words.
Prompting is not a substitute for probability measurements in large language models
Prompting is now a dominant method for evaluating the linguistic knowledge of large language models (LLMs).
Expectations over Unspoken Alternatives Predict Pragmatic Inferences
Here, we test a shared mechanism explaining SI rates within and across scales: context-driven expectations about the unspoken alternatives.
Language model acceptability judgements are not always robust to context
In this paper, we investigate the stability of language models' performance on targeted syntactic evaluations as we vary properties of the input context: the length of the context, the types of syntactic phenomena it contains, and whether or not there are violations of grammaticality.
On the Effect of Anticipation on Reading Times
We assess the effect of anticipation on reading by comparing how well surprisal and contextual entropy predict reading times on four naturalistic reading datasets: two self-paced and two eye-tracking.
Towards Human-Agent Communication via the Information Bottleneck Principle
Emergent communication research often focuses on optimizing task-specific utility as a driver for communication.
Assessing Group-level Gender Bias in Professional Evaluations: The Case of Medical Student End-of-Shift Feedback
Although approximately 50% of medical school graduates today are women, female physicians tend to be underrepresented in senior positions, make less money than their male counterparts and receive fewer promotions.
When Does Syntax Mediate Neural Language Model Performance? Evidence from Dropout Probes
Recent causal probing literature reveals when language models and syntactic probes use similar representations.
Analyzing Wrap-Up Effects through an Information-Theoretic Lens
Numerous analyses of reading time (RT) data have been implemented -- all in an effort to better understand the cognitive processes driving reading comprehension.
Revisiting the Uniform Information Density Hypothesis
The uniform information density (UID) hypothesis posits a preference among language users for utterances structured such that information is distributed uniformly across a signal.
Controlled Evaluation of Grammatical Knowledge in Mandarin Chinese Language Models
We find suggestive evidence that structural supervision helps with representing syntactic state across intervening content and improves performance in low-data settings, suggesting that the benefits of hierarchical inductive biases in acquiring dependency relationships may extend beyond English.
Scalable pragmatic communication via self-supervision
Models of context-sensitive communication often use the Rational Speech Act framework (RSA; Frank & Goodman, 2012), which formulates listeners and speakers in a cooperative reasoning process.
A Targeted Assessment of Incremental Processing in Neural Language Models and Humans
We present a targeted, scaled-up comparison of incremental processing in humans and neural language models by collecting by-word reaction time data for sixteen different syntactic test suites across a range of structural phenomena.
Structural Guidance for Transformer Language Models
Here we study whether structural guidance leads to more human-like systematic linguistic generalization in Transformer language models without resorting to pre-training on very large amounts of data.
What if This Modified That? Syntactic Interventions via Counterfactual Embeddings
Neural language models exhibit impressive performance on a variety of tasks, but their internal reasoning may be difficult to understand.
Learning Evolved Combinatorial Symbols with a Neuro-symbolic Generative Model
Humans have the ability to rapidly understand rich combinatorial concepts from limited data.
Investigating Novel Verb Learning in BERT: Selectional Preference Classes and Alternation-Based Syntactic Generalization
Previous studies investigating the syntactic abilities of deep learning models have not targeted the relationship between the strength of the grammatical generalization and the amount of evidence to which the model is exposed during training.
Cloze Distillation: Improving Neural Language Models with Human Next-Word Prediction
Contemporary autoregressive language models (LMs) trained purely on corpus data have been shown to capture numerous features of human incremental processing.
Structural Supervision Improves Few-Shot Learning and Syntactic Generalization in Neural Language Models
Humans can learn structural properties about a word from minimal experience, and deploy their learned syntactic representations uniformly in different grammatical contexts.
Bridging Information-Seeking Human Gaze and Machine Reading Comprehension
In this work, we analyze how human gaze during reading comprehension is conditioned on the given reading comprehension question, and whether this signal can be beneficial for machine reading comprehension.
SyntaxGym: An Online Platform for Targeted Evaluation of Language Models
Targeted syntactic evaluations have yielded insights into the generalizations learned by neural network language models.
On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behavior
Human reading behavior is tuned to the statistics of natural language: the time it takes human subjects to read a word can be predicted from estimates of the word's probability in context.
STARC: Structured Annotations for Reading Comprehension
We present STARC (Structured Annotations for Reading Comprehension), a new annotation framework for assessing reading comprehension with multiple choice questions.
Linking artificial and human neural representations of language
Through further task ablations and representational analyses, we find that tasks which produce syntax-light representations yield significant improvements in brain decoding performance.
Representation of Constituents in Neural Language Models: Coordination Phrase as a Case Study
We assess whether different neural language models trained on English and French represent phrase-level number and gender features, and use those features to drive downstream expectations.
Hierarchical Representation in Neural Language Models: Suppression and Recovery of Expectations
Deep learning sequence models have led to a marked increase in performance for a range of Natural Language Processing tasks, but it remains an open question whether they are able to induce proper hierarchical generalizations for representing natural language from linear input alone.
Language Learning and Processing in People and Machines
We explain the current computational models of language acquisition, their limitations, and how the insights from these models can be incorporated into NLP applications.
What Syntactic Structures block Dependencies in RNN Language Models?
Here, we provide new evidence that RNN language models are sensitive to hierarchical syntactic structure by investigating the filler--gap dependency and constraints on it, known as syntactic islands.
Availability-Based Production Predicts Speakers' Real-time Choices of Mandarin Classifiers
When the upcoming noun is less predictable, the use of a more specific classifier would reduce surprisal at the noun thus potentially facilitate comprehension (predicted by Uniform Information Density, Levy & Jaeger, 2007), but the use of that more specific classifier may be dispreferred from a production standpoint if accessing the general classifier is always available (predicted by Availability-Based Production; Bock, 1987; Ferreira & Dell, 2000).
Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic State
We deploy the methods of controlled psycholinguistic experimentation to shed light on the extent to which the behavior of neural network language models reflects incremental representations of syntactic state.
Structural Supervision Improves Learning of Non-Local Grammatical Dependencies
State-of-the-art LSTM language models trained on large corpora learn sequential contingencies in impressive detail and have been shown to acquire a number of non-local grammatical dependencies with some success.
What do RNN Language Models Learn about Filler--Gap Dependencies?
RNN language models have achieved state-of-the-art perplexity results and have proven useful in a suite of NLP tasks, but it is as yet unclear what syntactic generalizations they learn.
Comparing Models of Associative Meaning: An Empirical Investigation of Reference in Simple Language Games
These results shed light on the nature of the lexical resources that speakers and listeners can bring to bear in achieving reference through associative meaning alone.
RNNs as psycholinguistic subjects: Syntactic state and grammatical dependency
Recurrent neural networks (RNNs) are the state of the art in sequence modeling for natural language.
What do RNN Language Models Learn about Filler-Gap Dependencies?
RNN language models have achieved state-of-the-art perplexity results and have proven useful in a suite of NLP tasks, but it is as yet unclear what syntactic generalizations they learn.
Comparing Theories of Speaker Choice Using a Model of Classifier Production in Mandarin Chinese
Speakers often have more than one way to express the same meaning.
Word learning and the acquisition of syntactic--semantic overhypotheses
Children learning their first language face multiple problems of induction: how to learn the meanings of words, and how to build meaningful phrases from those words according to syntactic rules.
Assessing Language Proficiency from Eye Movements in Reading
We present a novel approach for determining learners' second language proficiency which utilizes behavioral traces of eye movements during reading.
A Statistical Comparison of Some Theories of NP Word Order
A frequent object of study in linguistic typology is the order of elements {demonstrative, adjective, numeral, noun} in the noun phrase.
Noisy-context surprisal as a human sentence processing cost model
We use the noisy-channel theory of human sentence comprehension to develop an incremental processing cost model that unifies and extends key features of expectation-based and memory-based models.
Data-driven learning of symbolic constraints for a log-linear model in a phonological setting
We propose a non-parametric Bayesian model for learning and weighting symbolically-defined constraints to populate a log-linear model.
