Search Results for author:
Found 22 papers, 12 papers with code
Training Text-to-Text Transformers with Privacy Guarantees
We focus on T5 and show that by using recent advances in JAX and XLA we can train models with DP that do not suffer a large drop in pre-training utility, nor in training speed, and can still be fine-tuned to high accuracies on downstream tasks (e. g.
MiTTenS: A Dataset for Evaluating Gender Mistranslation
Translation systems, including foundation models capable of translation, can produce errors that result in gender mistranslation, and such errors can be especially harmful.
Low-Rank Adaptation for Multilingual Summarization: An Empirical Study
Although the advancements of pre-trained Large Language Models have significantly accelerated recent progress in NLP, their ever-increasing size poses significant challenges for conventional fine-tuning, especially in memory-intensive tasks.
Dissecting Recall of Factual Associations in Auto-Regressive Language Models
Given a subject-relation query, we study how the model aggregates information about the subject and relation to predict the correct attribute.
Scaling Vision Transformers to 22 Billion Parameters
The scaling of Transformers has driven breakthrough capabilities for language models.
Ranked #1 on
Zero-Shot Transfer Image Classification
on ObjectNet
Simple Recurrence Improves Masked Language Models
In this work, we explore whether modeling recurrence into the Transformer architecture can both be beneficial and efficient, by building an extremely simple recurrent module into the Transformer.
Scaling Up Models and Data with $\texttt{t5x}$ and $\texttt{seqio}$
Recent neural network-based language models have benefited greatly from scaling up the size of training datasets and the number of parameters in the models themselves.
Diagnosing AI Explanation Methods with Folk Concepts of Behavior
We posit that folk concepts of behavior provide us with a "language" that humans understand behavior with.
Autoregressive Diffusion Models
We introduce Autoregressive Diffusion Models (ARDMs), a model class encompassing and generalizing order-agnostic autoregressive models (Uria et al., 2014) and absorbing discrete diffusion (Austin et al., 2021), which we show are special cases of ARDMs under mild assumptions.
The MultiBERTs: BERT Reproductions for Robustness Analysis
Experiments with pre-trained models such as BERT are often based on a single checkpoint.
The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?
There is a recent surge of interest in using attention as explanation of model predictions, with mixed evidence on whether attention can be used as such.
The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models
We present the Language Interpretability Tool (LIT), an open-source platform for visualization and understanding of NLP models.
We Need to Talk About Random Splits
We argue that random splits, like standard splits, lead to overly optimistic performance estimates.
Joey NMT: A Minimalist NMT Toolkit for Novices
We present Joey NMT, a minimalist neural machine translation toolkit based on PyTorch that is specifically designed for novices.
Interpretable Neural Predictions with Differentiable Binary Variables
The success of neural networks comes hand in hand with a desire for more interpretability.
Modeling Latent Sentence Structure in Neural Machine Translation
Recently it was shown that linguistic structure predicted by a supervised parser can be beneficial for neural machine translation (NMT).
Jump to better conclusions: SCAN both left and right
Lake and Baroni (2018) recently introduced the SCAN data set, which consists of simple commands paired with action sequences and is intended to test the strong generalization abilities of recurrent sequence-to-sequence models.
Exploiting Semantics in Neural Machine Translation with Graph Convolutional Networks
Semantic representations have long been argued as potentially useful for enforcing meaning preservation and improving generalization performance of machine translation methods.
Ranked #8 on
Machine Translation
on WMT2016 English-German
Graph Convolutional Encoders for Syntax-aware Neural Machine Translation
We present a simple and effective approach to incorporating syntactic structure into neural attention-based encoder-decoder models for machine translation.
All Fragments Count in Parser Evaluation
PARSEVAL, the default paradigm for evaluating constituency parsers, calculates parsing success (Precision/Recall) as a function of the number of matching labeled brackets across the test set.
