Search Results for author:
Found 23 papers, 17 papers with code
Automatic Construction of Morphologically Motivated Translation Models for Highly Inflected, Low-Resource Languages
Statistical Machine Translation (SMT) of highly inflected, low-resource languages suffers from the problem of low bitext availability, which is exacerbated by large inflectional paradigms.
Model Editing with Canonical Examples
The Backpack defines a large bank of sense vectors--a decomposition of the different uses of each word--which are weighted and summed to form the output logits of the model.
Character-level Chinese Backpack Language Models
The Backpack is a Transformer alternative shown to improve interpretability in English language modeling by decomposing predictions into a weighted sum of token sense components.
Closing the Curious Case of Neural Text Degeneration
We provide a theoretical explanation for the effectiveness of the truncation sampling by proving that truncation methods that discard tokens below some probability threshold (the most common type of truncation) can guarantee that all sampled tokens have nonzero true probability.
Lost in the Middle: How Language Models Use Long Contexts
While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context.
Backpack Language Models
We can interpret a sense vector by inspecting its (non-contextual, linear) projection onto the output space, and intervene on these interpretable hooks to change the model's behavior in predictable ways.
JamPatoisNLI: A Jamaican Patois Natural Language Inference Dataset
While our work, along with previous work, shows that transfer from these models to low-resource languages that are unrelated to languages in their training set is not very effective, we would expect stronger results from transfer to creoles.
Truncation Sampling as Language Model Desmoothing
In this light, truncation algorithms aim to perform desmoothing, estimating a subset of the support of the true distribution.
Ensembles and Cocktails: Robust Finetuning for Natural Language Generation
When finetuning a pretrained language model for natural language generation tasks, one is currently faced with a tradeoff.
Conditional probing: measuring usable information beyond a baseline
Probing experiments investigate the extent to which neural representations make properties -- like part-of-speech -- predictable.
On the Opportunities and Risks of Foundation Models
AI is undergoing a paradigm shift with the rise of models (e. g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.
Refining Targeted Syntactic Evaluation of Language Models
Targeted syntactic evaluation of subject-verb number agreement in English (TSE) evaluates language models' syntactic knowledge using hand-crafted minimal pairs of sentences that differ only in the main verb's conjugation.
Probing artificial neural networks: insights from neuroscience
A major challenge in both neuroscience and machine learning is the development of useful tools for understanding complex information processing systems.
RNNs can generate bounded hierarchical languages with optimal memory
Recurrent neural networks empirically generate natural language with high syntactic fidelity.
The EOS Decision and Length Extrapolation
Extrapolation to unseen sequence lengths is a challenge for neural generative models of language.
Finding Universal Grammatical Relations in Multilingual BERT
Recent work has found evidence that Multilingual BERT (mBERT), a transformer-based multilingual masked language model, is capable of zero-shot cross-lingual transfer, suggesting that some aspects of its representations are shared cross-lingually.
Designing and Interpreting Probes with Control Tasks
The selectivity of a probe puts linguistic task accuracy in context with the probe's capacity to memorize from word types.
A Structural Probe for Finding Syntax in Word Representations
Recent work has improved our ability to detect linguistic knowledge in word representations.
Simple, Fast, Accurate Intent Classification and Slot Labeling for Goal-Oriented Dialogue Systems
With the advent of conversational assistants, like Amazon Alexa, Google Now, etc., dialogue systems are gaining a lot of traction, especially in industrial setting.
A Distributional and Orthographic Aggregation Model for English Derivational Morphology
Modeling derivational morphology to generate words with particular semantics is useful in many text generation tasks, such as machine translation or abstractive question answering.
Learning Translations via Images with a Massively Multilingual Image Dataset
To facilitate research on the task, we introduce a large-scale multilingual corpus of images, each labeled with the word it represents.
XNMT: The eXtensible Neural Machine Translation Toolkit
In this paper we describe the design of XNMT and its experiment configuration system, and demonstrate its utility on the tasks of machine translation, speech recognition, and multi-tasked machine translation/parsing.
Learning Translations via Matrix Completion
Bilingual Lexicon Induction is the task of learning word translations without bilingual parallel corpora.
