Search Results for author:
Found 17 papers, 9 papers with code
From Language Models over Tokens to Language Models over Characters
Modern language models are internally -- and mathematically -- distributions over token strings rather than \emph{character} strings, posing numerous challenges for programmers building user applications on top of them.
Reframing linguistic bootstrapping as joint inference using visually-grounded grammar induction models
Here, we argue that they are instead both contingent on a more general learning strategy for language acquisition: joint learning.
Correlation Does Not Imply Compensation: Complexity and Irregularity in the Lexicon
It has been claimed that within a language, morphologically irregular words are more likely to be phonotactically simple and morphologically regular words are more likely to be phonotactically complex.
The Stable Entropy Hypothesis and Entropy-Aware Decoding: An Analysis and Algorithm for Robust Natural Language Generation
State-of-the-art language generation models can degenerate when applied to open-ended generation problems such as text completion, story generation, or dialog modeling.
Evaluating Distributional Distortion in Neural Language Modeling
To address this gap, we develop a controlled evaluation scheme which uses generative models trained on natural data as artificial languages from which we can exactly compute sequence probabilities.
Systematic Generalization with Edge Transformers
Recent research suggests that systematic generalization in natural language understanding remains a challenge for state-of-the-art neural models such as Transformers and Graph Neural Networks.
Compositional Generalization in Dependency Parsing
To test compositional generalization in semantic parsing, Keysers et al. (2020) introduced Compositional Freebase Queries (CFQ).
Linguistic Dependencies and Statistical Dependence
Are pairs of words that tend to occur together also likely to stand in a linguistic dependency?
Characterizing Idioms: Conventionality and Contingency
Second, the non-canonical meanings of words in an idiom are contingent on the presence of other words in the idiom.
Jointly Learning Truth-Conditional Denotations and Groundings using Parallel Attention
We present a model that jointly learns the denotations of words together with their groundings using a truth-conditional semantics.
Recursive Top-Down Production for Sentence Generation with Latent Trees
We model the recursive production property of context-free grammars for natural and synthetic languages.
Exploiting Syntactic Structure for Better Language Modeling: A Syntactic Distance Approach
It is commonly believed that knowledge of syntactic structure should improve language modeling.
Probing Linguistic Systematicity
Recently, there has been much interest in the question of whether deep natural language understanding models exhibit systematicity; generalizing such that units like words make consistent contributions to the meaning of the sentences in which they appear.
CLOSURE: Assessing Systematic Generalization of CLEVR Models
In this work, we study how systematic the generalization of such models is, that is to which extent they are capable of handling novel combinations of known linguistic constructs.
Morphological Irregularity Correlates with Frequency
We present a study of morphological irregularity.
Grammar Induction for Minimalist Grammars using Variational Bayesian Inference : A Technical Report
The following technical report presents a formal approach to probabilistic minimalist grammar parameter estimation.
A generalized parsing framework for Abstract Grammars
This technical report presents a general framework for parsing a variety of grammar formalisms.
