Search Results for author:
Found 40 papers, 16 papers with code
EASE: Extractive-Abstractive Summarization End-to-End using the Information Bottleneck Principle
To achieve the best of both worlds, we propose EASE, an extractive-abstractive framework that generates concise abstractive summaries that can be traced back to an extractive summary.
Non-autoregressive Translation with Disentangled Context Transformer
State-of-the-art neural machine translation models generate a translation from left to right and every step is conditioned on the previously generated tokens.
Ranked #54 on
Machine Translation
on WMT2014 English-German
JPEG-LM: LLMs as Image Generators with Canonical Codec Representations
Evaluation of image generation shows that this simple and straightforward approach is more effective than pixel-based modeling and sophisticated vector quantization baselines (on which our method yields a 31% reduction in FID).
David helps Goliath: Inference-Time Collaboration Between Small Specialized and Large General Diffusion LMs
Diffusion-based language models are emerging as a promising alternative to autoregressive LMs: they approach the competence of autoregressive LMs while offering nuanced controllability at inference time.
Dictionary-based Phrase-level Prompting of Large Language Models for Machine Translation
Large language models (LLMs) demonstrate remarkable machine translation (MT) abilities via prompting, even though they were not explicitly trained for this task.
Representation Deficiency in Masked Language Modeling
In this work, we offer a new perspective on the consequence of such a discrepancy: We demonstrate empirically and theoretically that MLM pretraining allocates some model dimensions exclusively for representing $\texttt{[MASK]}$ tokens, resulting in a representation deficiency for real tokens and limiting the pretrained model's expressiveness when it is adapted to downstream data without $\texttt{[MASK]}$ tokens.
XLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Models
Large multilingual language models typically rely on a single vocabulary shared across 100+ languages.
In-context Examples Selection for Machine Translation
Large-scale generative models show an impressive ability to perform a wide range of Natural Language Processing (NLP) tasks using in-context learning, where a few examples are used to describe a task to the model.
Natural Language to Code Translation with Execution
In this work, we introduce execution result--based minimum Bayes risk decoding (MBR-EXEC) for program selection and show that it improves the few-shot performance of pretrained code models on natural-language-to-code tasks.
Ranked #38 on
Code Generation
on MBPP
A Review on Language Models as Knowledge Bases
Recently, there has been a surge of interest in the NLP community on the use of pretrained Language Models (LMs) as Knowledge Bases (KBs).
Discourse-Aware Soft Prompting for Text Generation
We show that soft-prompt based conditional text generation can be improved with simple and efficient methods that simulate modeling the discourse structure of human written text.
BitextEdit: Automatic Bitext Editing for Improved Low-Resource Machine Translation
Mined bitexts can contain imperfect translations that yield unreliable training signals for Neural Machine Translation (NMT).
Distributionally Robust Multilingual Machine Translation
Multilingual neural machine translation (MNMT) learns to translate multiple language pairs with a single model, potentially improving both the accuracy and the memory-efficiency of deployed models.
Prompting Contrastive Explanations for Commonsense Reasoning Tasks
Many commonsense reasoning NLP tasks involve choosing between one or more possible answers to a question or prompt based on knowledge that is often implicit.
EASE: Extractive-Abstractive Summarization with Explanations
Current abstractive summarization systems outperform their extractive counterparts, but their widespread adoption is inhibited by the inherent lack of interpretability.
Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog
Semantic parsing using sequence-to-sequence models allows parsing of deeper representations compared to traditional word tagging based models.
Detecting Hallucinated Content in Conditional Neural Sequence Generation
Neural sequence models can generate highly fluent sentences, but recent studies have also shown that they are also prone to hallucinate additional content not supported by the input.
Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation
Models pretrained with self-supervised objectives on large text corpora achieve state-of-the-art performance on English text summarization tasks.
DeLighT: Deep and Light-weight Transformer
We introduce a deep and light-weight transformer, DeLighT, that delivers similar or better performance than standard transformer-based models with significantly fewer parameters.
Ranked #1 on
Machine Translation
on WMT2016 English-French
Simple and Effective Retrieve-Edit-Rerank Text Generation
Retrieve-and-edit seq2seq methods typically retrieve an output from the training set and learn a model to edit it to produce the final output.
Pre-training via Paraphrasing
The objective noisily captures aspects of paraphrase, translation, multi-document summarization, and information retrieval, allowing for strong zero-shot performance on several tasks.
Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translation
For BART we get the best performance by freezing most of the model parameters, and adding extra positional embeddings.
Aligned Cross Entropy for Non-Autoregressive Machine Translation
This difficultly is compounded during training with cross entropy loss, which can highly penalize small shifts in word order.
Semi-Autoregressive Training Improves Mask-Predict Decoding
The recently proposed mask-predict decoding algorithm has narrowed the performance gap between semi-autoregressive machine translation models and the traditional left-to-right approach.
Non-Autoregressive Machine Translation with Disentangled Context Transformer
State-of-the-art neural machine translation models generate a translation from left to right and every step is conditioned on the previously generated tokens.
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
We evaluate a number of noising approaches, finding the best performance by both randomly shuffling the order of the original sentences and using a novel in-filling scheme, where spans of text are replaced with a single mask token.
Ranked #3 on
Open-Domain Question Answering
on ELI5
Mask-Predict: Parallel Decoding of Conditional Masked Language Models
Most machine translation systems generate text autoregressively from left to right.
Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translation
We consider the problem of making machine translation more robust to character-level variation at the source side, such as typos.
Towards Controllable Story Generation
We present a general framework of analyzing existing story corpora to generate controllable and creative new stories.
Using Word Vectors to Improve Word Alignments for Low Resource Machine Translation
We present a method for improving word alignments using word similarities.
A Knowledge-Grounded Neural Conversation Model
We generalize the widely-used Seq2Seq approach by conditioning responses on both conversation history and external "facts", allowing the model to be versatile and applicable in an open-domain setting.
Learning and Optimization with Submodular Functions
Set functions, and specifically submodular set functions, characterize a wide variety of naturally occurring optimization problems, and the property of submodularity of set functions has deep theoretical consequences with wide ranging applications.
From Local Similarity to Global Coding: An Application to Image Classification
Exploiting the local similarity of a descriptor and its nearby bases, a global measure of association of a descriptor to all the bases is computed.
