Search Results for author:
Found 45 papers, 14 papers with code
When classifying grammatical role, BERT doesn’t care about word order... except when it matters
Because meaning can often be inferred from lexical semantics alone, word order is often a redundant cue in natural language.
An Information-Theoretic Characterization of Morphological Fusion
Linguistic typology generally divides synthetic languages into groups based on their morphological fusion.
Simple induction of (deterministic) probabilistic finite-state automata for phonotactics by stochastic gradient descent
We introduce a simple and highly general phonotactic learner which induces a probabilistic finite-state automaton from word-form data.
Investigating Information-Theoretic Properties of the Typology of Spatial Demonstratives
Using data from Nintemann et al. (2020), we explore the variability in complexity and informativity across spatial demonstrative systems using spatial deictic lexicons from 223 languages.
Estimating word co-occurrence probabilities from pretrained static embeddings using a log-bilinear model
We investigate how to use pretrained static word embeddings to deliver improved estimates of bilexical co-occurrence probabilities: conditional probabilities of one word given a single other word in a specific relationship.
Measuring Morphological Fusion Using Partial Information Decomposition
Morphological systems across languages vary when it comes to the relation between form and meaning.
Assessing Corpus Evidence for Formal and Psycholinguistic Constraints on Nonprojectivity
In Experiment 1, we compare the distribution of formal properties of crossing dependencies, such as gap degree, between real trees and baseline trees matched for rate of crossing dependencies and various other properties.
What Determines the Order of Verbal Dependents in Hindi? Effects of Efficiency in Comprehension and Production
In this work, we investigate whether the order and relative distance of preverbal dependents in Hindi, an SOV language, is affected by factors motivated by efficiency considerations during comprehension/production.
Mission: Impossible Language Models
Chomsky and others have very directly claimed that large language models (LLMs) are equally capable of learning languages that are possible and impossible for humans to learn.
Exploring the Sensitivity of LLMs' Decision-Making Capabilities: Insights from Prompt Variation and Hyperparameters
However, these studies have not always properly accounted for the sensitivity of LLMs' behavior to hyperparameters and variations in the prompt.
A Cross-Linguistic Pressure for Uniform Information Density in Word Order
Here, we ask whether a pressure for UID may have influenced word order patterns cross-linguistically.
A unified information-theoretic model of EEG signatures of human language processing
We advance an information-theoretic model of human language processing in the brain, in which incoming linguistic input is processed at two levels, in terms of a heuristic interpretation and in terms of error correction.
When classifying grammatical role, BERT doesn't care about word order... except when it matters
Because meaning can often be inferred from lexical semantics alone, word order is often a redundant cue in natural language.
Grammatical cues to subjecthood are redundant in a majority of simple clauses across languages
The conclusion is that grammatical cues such as word order are necessary to convey subjecthood and objecthood in a minority of naturally occurring transitive clauses; nevertheless, they can (a) provide an important source of redundancy and (b) are crucial for conveying intended meaning that cannot be inferred from the words alone, including descriptions of human interactions, where roles are often reversible (e. g., Ray helped Lu/Lu helped Ray), and expressing non-prototypical meanings (e. g., "The bone chewed the dog.
Sensitivity as a Complexity Measure for Sequence Classification Tasks
We introduce a theoretical framework for understanding and predicting the complexity of sequence classification tasks, using a novel extension of the theory of Boolean function sensitivity.
Deep Subjecthood: Higher-Order Grammatical Features in Multilingual BERT
Further examining the characteristics that our classifiers rely on, we find that features such as passive voice, animacy and case strongly correlate with classification decisions, suggesting that mBERT does not encode subjecthood purely syntactically, but that subjecthood embedding is continuous and dependent on semantic and discourse factors, as is proposed in much of the functional linguistics literature.
Predicting cross-linguistic adjective order with information gain
Languages vary in their placement of multiple adjectives before, after, or surrounding the noun, but they typically exhibit strong intra-language tendencies on the relative order of those adjectives (e. g., the preference for `big blue box' in English, `grande bo\^{i}te bleue' in French, and `alsund\={u}q al'azraq alkab\={\i}r' in Arabic).
Structural Supervision Improves Few-Shot Learning and Syntactic Generalization in Neural Language Models
Humans can learn structural properties about a word from minimal experience, and deploy their learned syntactic representations uniformly in different grammatical contexts.
What determines the order of adjectives in English? Comparing efficiency-based theories using dependency treebanks
The four theories we test are subjectivity (Scontras et al., 2017), information locality (Futrell, 2019), integration cost (Dyer, 2017), and information gain, which we introduce.
An information-theoretic account of semantic interference in word production
I present a computational-level model of semantic interference effects in word production.
Hierarchical Representation in Neural Language Models: Suppression and Recovery of Expectations
Deep learning sequence models have led to a marked increase in performance for a range of Natural Language Processing tasks, but it remains an open question whether they are able to induce proper hierarchical generalizations for representing natural language from linear input alone.
Language Learning and Processing in People and Machines
We explain the current computational models of language acquisition, their limitations, and how the insights from these models can be incorporated into NLP applications.
What Syntactic Structures block Dependencies in RNN Language Models?
Here, we provide new evidence that RNN language models are sensitive to hierarchical syntactic structure by investigating the filler--gap dependency and constraints on it, known as syntactic islands.
Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic State
We deploy the methods of controlled psycholinguistic experimentation to shed light on the extent to which the behavior of neural network language models reflects incremental representations of syntactic state.
Structural Supervision Improves Learning of Non-Local Grammatical Dependencies
State-of-the-art LSTM language models trained on large corpora learn sequential contingencies in impressive detail and have been shown to acquire a number of non-local grammatical dependencies with some success.
Do RNNs learn human-like abstract word order preferences?
We collect human acceptability ratings for our stimuli, in the first acceptability judgment experiment directly manipulating the predictors of syntactic alternations.
What do RNN Language Models Learn about Filler--Gap Dependencies?
RNN language models have achieved state-of-the-art perplexity results and have proven useful in a suite of NLP tasks, but it is as yet unclear what syntactic generalizations they learn.
RNNs as psycholinguistic subjects: Syntactic state and grammatical dependency
Recurrent neural networks (RNNs) are the state of the art in sequence modeling for natural language.
What do RNN Language Models Learn about Filler-Gap Dependencies?
RNN language models have achieved state-of-the-art perplexity results and have proven useful in a suite of NLP tasks, but it is as yet unclear what syntactic generalizations they learn.
A Statistical Comparison of Some Theories of NP Word Order
A frequent object of study in linguistic typology is the order of elements {demonstrative, adjective, numeral, noun} in the noun phrase.
The Natural Stories Corpus
It is now a common practice to compare models of human language processing by predicting participant reactions (such as reading times) to corpora consisting of rich naturalistic linguistic materials.
Noisy-context surprisal as a human sentence processing cost model
We use the noisy-channel theory of human sentence comprehension to develop an incremental processing cost model that unifies and extends key features of expectation-based and memory-based models.
A Generative Model of Phonotactics
We present a probabilistic model of phonotactics, the set of well-formed phoneme sequences in a language.
Memory access during incremental sentence processing causes reading time latency
Studies on the role of memory as a predictor of reading time latencies (1) differ in their predictions about when memory effects should occur in processing and (2) have had mixed results, with strong positive effects emerging from isolated constructed stimuli and weak or even negative effects emerging from naturally-occurring stimuli.
Response to Liu, Xu, and Liang (2015) and Ferrer-i-Cancho and Gómez-Rodríguez (2015) on Dependency Length Minimization
We address recent criticisms (Liu et al., 2015; Ferrer-i-Cancho and G\'omez-Rodr\'iguez, 2015) of our work on empirical evidence of dependency length minimization across languages (Futrell et al., 2015).
