Search Results for author:
Found 20 papers, 12 papers with code
A Database for Modal Semantic Typology
This paper introduces a database for crosslinguistic modal semantics.
Minimization of Boolean Complexity in In-Context Concept Learning
What factors contribute to the relative success and corresponding difficulties of in-context learning for Large Language Models (LLMs)?
Filtered Corpus Training (FiCT) Shows that Language Models can Generalize from Indirect Evidence
This paper introduces Filtered Corpus Training, a method that trains language models (LMs) on corpora with certain linguistic constructions filtered out from the training data, and uses it to measure the ability of LMs to perform linguistic generalization on the basis of indirect evidence.
Targeted Multilingual Adaptation for Low-resource Language Families
To test this approach more rigorously, we systematically study best practices for adapting a pre-trained model to a language family.
The Impact of Syntactic and Semantic Proximity on Machine Translation with Back-Translation
Unsupervised on-the-fly back-translation, in conjunction with multilingual pretraining, is the dominant method for unsupervised neural machine translation.
Embedding structure matters: Comparing methods to adapt multilingual vocabularies to new languages
Pre-trained multilingual language models underpin a large portion of modern NLP tools outside of English.
Evaluating Transformer's Ability to Learn Mildly Context-Sensitive Languages
Despite the fact that Transformers perform well in NLP tasks, recent studies suggest that self-attention is theoretically limited in learning even some regular and context-free languages.
The Weighted Möbius Score: A Unified Framework for Feature Attribution
Feature attribution aims to explain the reasoning behind a black-box model's prediction by identifying the impact of each feature on the prediction.
Testing Pre-trained Language Models' Understanding of Distributivity via Causal Mediation Analysis
To what extent do pre-trained language models grasp semantic knowledge regarding the phenomenon of distributivity?
Probing for Understanding of English Verb Classes and Alternations in Large Pre-trained Language Models
We investigate the extent to which verb alternation classes, as described by Levin (1993), are encoded in the embeddings of Large Pre-trained Language Models (PLMs) such as BERT, RoBERTa, ELECTRA, and DeBERTa using selectively constructed diagnostic classifiers for word and sentence-level prediction tasks.
Learning to translate by learning to communicate
We formulate and test a technique to use Emergent Communication (EC) with a pre-trained multilingual model to improve on modern Unsupervised NMT systems, especially for low-resource languages.
Language Models Use Monotonicity to Assess NPI Licensing
We investigate the semantic knowledge of language models (LMs), focusing on (1) whether these LMs create categories of linguistic environments based on their semantic monotonicity properties, and (2) whether these categories play a similar role in LMs as in human language understanding, using negative polarity item licensing as a case study.
A multilabel approach to morphosyntactic probing
We introduce a multilabel probing task to assess the morphosyntactic representations of word embeddings from multilingual language models.
A Masked Segmental Language Model for Unsupervised Natural Language Segmentation
Segmentation remains an important preprocessing step both in languages where "words" or other important syntactic/semantic units (like morphemes) are not clearly delineated by white space, as well as when dealing with continuous speech data, where there is often no meaningful pause between words.
Bilingual alignment transfers to multilingual alignment for unsupervised parallel text mining
To study this theory, we design unsupervised models trained on unpaired sentences and single-pair supervised models trained on bitexts, both based on the unsupervised language model XLM-R with its parameters frozen.
Linguistically-Informed Transformations (LIT): A Method for Automatically Generating Contrast Sets
Although large-scale pretrained language models, such as BERT and RoBERTa, have achieved superhuman performance on in-distribution test sets, their performance suffers on out-of-distribution test sets (e. g., on contrast sets).
On the Spontaneous Emergence of Discrete and Compositional Signals
We propose a general framework to study language emergence through signaling games with neural agents.
Paying Attention to Function Words
All natural languages exhibit a distinction between content words (like nouns and adjectives) and function words (like determiners, auxiliaries, prepositions).
Neural Models of the Psychosemantics of `Most'
How are the meanings of linguistic expressions related to their use in concrete cognitive tasks?
Some of Them Can be Guessed! Exploring the Effect of Linguistic Context in Predicting Quantifiers
We study the role of linguistic context in predicting quantifiers (`few', `all').
