Search Results for author:
Found 30 papers, 13 papers with code
LIEDER: Linguistically-Informed Evaluation for Discourse Entity Recognition
Discourse Entity (DE) recognition is the task of identifying novel and known entities introduced within a text.
How Abstract Is Linguistic Generalization in Large Language Models? Experiments with Argument Structure
We find that LLMs perform well in generalizing the distribution of a novel noun argument between related contexts that were seen during pre-training (e. g., the active object and passive subject of the verb spray), succeeding by making use of the semantically-organized structure of the embedding space for word embeddings.
False perspectives on human language: why statistics needs linguistics
A sharp tension exists about the nature of human language between two opposite parties: those who believe that statistical surface distributions, in particular using measures like surprisal, provide a better understanding of language processing, vs. those who believe that discrete hierarchical structures implementing linguistic information such as syntactic ones are a better tool.
How poor is the stimulus? Evaluating hierarchical generalization in neural networks trained on child-directed speech
When acquiring syntax, children consistently choose hierarchical rules over competing non-hierarchical possibilities.
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
Formal Language Recognition by Hard Attention Transformers: Perspectives from Circuit Complexity
This paper analyzes three formal models of Transformer encoders that differ in the form of their self-attention mechanism: unique hard attention (UHAT); generalized unique hard attention (GUHAT), which generalizes UHAT; and averaging hard attention (AHAT).
Coloring the Blank Slate: Pre-training Imparts a Hierarchical Inductive Bias to Sequence-to-sequence Models
We find that pre-trained seq2seq models generalize hierarchically when performing syntactic transformations, whereas models trained from scratch on syntactic transformations do not.
Do Language Models Learn Position-Role Mappings?
How is knowledge of position-role mappings in natural language learned?
Transformers Generalize Linearly
Natural language exhibits patterns of hierarchically governed dependencies, in which relations between words are sensitive to syntactic structure rather than linear ordering.
Sequence-to-Sequence Networks Learn the Meaning of Reflexive Anaphora
Reflexive anaphora present a challenge for semantic interpretation: their meaning varies depending on context in a way that appears to require abstract variables.
Probabilistic Predictions of People Perusing: Evaluating Metrics of Language Model Performance for Psycholinguistic Modeling
By positing a relationship between naturalistic reading times and information-theoretic surprisal, surprisal theory (Hale, 2001; Levy, 2008) provides a natural interface between language models and psycholinguistic models.
Does syntax need to grow on trees? Sources of hierarchical inductive bias in sequence-to-sequence networks
We investigate which architectural factors affect the generalization behavior of neural sequence-to-sequence models trained on two syntactic tasks, English question formation and English tense reinflection.
Finding Hierarchical Structure in Neural Stacks Using Unsupervised Parsing
Neural network architectures have been augmented with differentiable stacks in order to introduce a bias toward learning hierarchy-sensitive regularities.
Finding Syntactic Representations in Neural Stacks
Neural network architectures have been augmented with differentiable stacks in order to introduce a bias toward learning hierarchy-sensitive regularities.
Open Sesame: Getting Inside BERT's Linguistic Knowledge
How and to what extent does BERT encode syntactically-sensitive hierarchical information or positionally-sensitive linear information?
Detecting Syntactic Change Using a Neural Part-of-Speech Tagger
Thus, our results suggest that our tagger is implicitly learning to model syntactic change in American English over the course of the 19th, 20th, and early 21st centuries.
Syntax-aware Neural Semantic Role Labeling with Supertags
We introduce a new syntax-aware model for dependency-based semantic role labeling that outperforms syntax-agnostic models for English and Spanish.
Context-Free Transductions with Neural Stacks
This paper analyzes the behavior of stack-augmented recurrent neural network (RNN) models.
End-to-end Graph-based TAG Parsing with Neural Networks
We present a graph-based Tree Adjoining Grammar (TAG) parser that uses BiLSTMs, highway connections, and character-level CNNs.
Revisiting the poverty of the stimulus: hierarchical generalization without a hierarchical bias in recurrent neural networks
We examine this proposal using recurrent neural networks (RNNs), which are not constrained in such a way.
