Search Results for author:
Found 60 papers, 31 papers with code
Improving Text-to-Text Pre-trained Models for the Graph-to-Text Task
Converting a knowledge graph or sub-graph to natural text is useful when answering questions based on a knowledge base.
Generating Biographies on Wikipedia: The Impact of Gender Bias on the Retrieval-Based Generation of Women Biographies
To assess the impact of available web evidence on the output text, we compare the performance of our approach when generating biographies about women (for which less information is available on the web) vs. biographies generally.
Findings of the AmericasNLP 2021 Shared Task on Open Machine Translation for Indigenous Languages of the Americas
This paper presents the results of the 2021 Shared Task on Open Machine Translation for Indigenous Languages of the Americas.
Facebook AI’s WMT21 News Translation Task Submission
We describe Facebook’s multilingual model submission to the WMT2021 shared task on news translation.
Findings of the WMT 2021 Shared Task on Large-Scale Multilingual Machine Translation
There were a total of 10 participating teams for the tasks, with a total of 151 intermediate model submissions and 13 final models.
Findings of the 2021 Conference on Machine Translation (WMT21)
This paper presents the results of the newstranslation task, the multilingual low-resourcetranslation for Indo-European languages, thetriangular translation task, and the automaticpost-editing task organised as part of the Con-ference on Machine Translation (WMT) 2021. In the news task, participants were asked tobuild machine translation systems for any of10 language pairs, to be evaluated on test setsconsisting mainly of news stories.
Facebook AI’s WMT20 News Translation Task Submission
We approach the low resource problem using two main strategies, leveraging all available data and adapting the system to the target news domain.
Effective Long-Context Scaling of Foundation Models
We also examine the impact of various design choices in the pretraining process, including the data mix and the training curriculum of sequence lengths -- our ablation experiments suggest that having abundant long texts in the pretrain dataset is not the key to achieving strong performance, and we empirically verify that long context continual pretraining is more efficient and similarly effective compared to pretraining from scratch with long sequences.
Ngambay-French Neural Machine Translation (sba-Fr)
However, a guided approach for data gathering can produce bitext data for many Chadian language translation pairs with well-known languages that have ample data.
Llama 2: Open Foundation and Fine-Tuned Chat Models
In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters.
Ranked #2 on
Question Answering
on PubChemQA
Revisiting Machine Translation for Cross-lingual Classification
Machine Translation (MT) has been widely used for cross-lingual classification, either by translating the test set into English and running inference with a monolingual model (translate-test), or translating the training set into the target languages and finetuning a multilingual model (translate-train).
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions.
RQUGE: Reference-Free Metric for Evaluating Question Generation by Answering the Question
We demonstrate that RQUGE has a higher correlation with human judgment without relying on the reference question.
No Language Left Behind: Scaling Human-Centered Machine Translation
Driven by the goal of eradicating language barriers on a global scale, machine translation has solidified itself as a key focus of artificial intelligence research today.
Ranked #1 on
Machine Translation
on IWSLT2017 French-English
(SacreBLEU metric)
A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translation
We focus on two questions: 1) How can pre-trained models be used for languages not included in the initial pre-training?
Generating Full Length Wikipedia Biographies: The Impact of Gender Bias on the Retrieval-Based Generation of Women Biographies
To assess the impact of available web evidence on the output text, we compare the performance of our approach when generating biographies about women (for which less information is available on the web) vs. biographies generally.
Reasoning over Public and Private Data in Retrieval-Based Systems
We first define the PUBLIC-PRIVATE AUTOREGRESSIVE INFORMATION RETRIEVAL (PAIR) privacy framework for the novel retrieval setting over multiple privacy scopes.
Ranked #1 on
Multi-hop Question Answering
on ConcurrentQA
Alternative Input Signals Ease Transfer in Multilingual Machine Translation
Further, we find that incorporating alternative inputs via self-ensemble can be particularly effective when training set is small, leading to +5 BLEU when only 5% of the total training data is accessible.
Tricks for Training Sparse Translation Models
Multi-task learning with an unbalanced data distribution skews model learning towards high resource tasks, especially when model capacity is fixed and fully shared across all tasks.
Facebook AI WMT21 News Translation Task Submission
We describe Facebook's multilingual model submission to the WMT2021 shared task on news translation.
The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation
One of the biggest challenges hindering progress in low-resource and multilingual machine translation is the lack of good evaluation benchmarks.
Not All Memories are Created Equal: Learning to Forget by Expiring
We demonstrate that Expire-Span can help models identify and retain critical information and show it can achieve strong performance on reinforcement learning tasks specifically designed to challenge this functionality.
Ranked #4 on
Language Modelling
on enwik8
AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages
Continued pretraining offers improvements, with an average accuracy of 44. 05%.
Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog
Semantic parsing using sequence-to-sequence models allows parsing of deeper representations compared to traditional word tagging based models.
CycleDRUMS: Automatic Drum Arrangement For Bass Lines Using CycleGAN
The two main research threads in computer-based music generation are: the construction of autonomous music-making systems, and the design of computer-based environments to assist musicians.
Automatic Music Production Using Generative Adversarial Networks
In this work, we propose a novel framework for $\textit{automatic music arrangement from raw audio in the frequency domain}$.
Not All Memories are Created Equal: Learning to Expire
We demonstrate that Expire-Span can help models identify and retain critical information and show it can achieve state of the art results on long-context language modeling, reinforcement learning, and algorithmic tasks.
Human Evaluation of Spoken vs. Visual Explanations for Open-Domain QA
While research on explaining predictions of open-domain QA systems (ODQA) to users is gaining momentum, most works have failed to evaluate the extent to which explanations improve user trust.
Facebook AI's WMT20 News Translation Task Submission
We approach the low resource problem using two main strategies, leveraging all available data and adapting the system to the target news domain.
Multilingual AMR-to-Text Generation
Generating text from structured data is challenging because it requires bridging the gap between (i) structure and natural language (NL) and (ii) semantically underspecified input and fully specified NL output.
Generating Fact Checking Briefs
Fact checking at scale is difficult -- while the number of active fact checking websites is growing, it remains too small for the needs of the contemporary media ecosystem.
Beyond English-Centric Multilingual Machine Translation
Existing work in translation demonstrated the potential of massively multilingual machine translation by training a single model able to translate between any pair of languages.
Nearest Neighbor Machine Translation
We introduce $k$-nearest-neighbor machine translation ($k$NN-MT), which predicts tokens with a nearest neighbor classifier over a large datastore of cached examples, using representations from a neural translation model for similarity search.
KILT: a Benchmark for Knowledge Intensive Language Tasks
We test both task-specific and general baselines, evaluating downstream performance in addition to the ability of the models to provide provenance.
Ranked #3 on
Entity Linking
on KILT: WNED-CWEB
Multilingual Translation with Extensible Multilingual Pretraining and Finetuning
Recent work demonstrates the potential of multilingual pretraining of creating one model that can be used for various tasks in different languages.
Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions
We present our view of what is necessary to build an engaging open-domain conversational agent: covering the qualities of such an agent, the pieces of the puzzle that have been built so far, and the gaping holes we have not filled yet.
Multi-Dimensional Gender Bias Classification
We show our classifiers prove valuable for a variety of important applications, such as controlling for gender bias in generative models, detecting gender bias in arbitrary text, and shed light on offensive language in terms of genderedness.
MUSS: Multilingual Unsupervised Sentence Simplification by Mining Paraphrases
Progress in sentence simplification has been hindered by a lack of labeled parallel simplification data, particularly in languages other than English.
Ranked #2 on
Text Simplification
on ASSET
Augmenting Transformers with KNN-Based Composite Memory for Dialogue
Various machine learning tasks can benefit from access to external information of different modalities, such as text and images.
Training with Quantization Noise for Extreme Model Compression
A standard solution is to train networks with Quantization Aware Training, where the weights are quantized during training and the gradients approximated with the Straight-Through Estimator.
Addressing Some Limitations of Transformers with Feedback Memory
Transformers have been successfully applied to sequential, auto-regressive tasks despite being feedforward networks.
Generating Interactive Worlds with Text
We show that the game environments created with our approach are cohesive, diverse, and preferred by human evaluators compared to other machine learning based world construction algorithms.
Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation
Models often easily learn biases present in the training data, and their predictions directly reflect this bias.
Using Local Knowledge Graph Construction to Scale Seq2Seq Models to Multi-Document Inputs
Query-based open-domain NLP tasks require information synthesis from long and diverse web results.
Ranked #4 on
Open-Domain Question Answering
on ELI5
Reducing Transformer Depth on Demand with Structured Dropout
Overparameterized transformer networks have obtained state of the art results in various natural language processing tasks, such as machine translation, language modeling, and question answering.
Ranked #5 on
Open-Domain Question Answering
on ELI5
Augmenting Transformers with KNN-Based Composite Memory
Various machine learning tasks can benefit from access to external information of different modalities, such as text and images.
ELI5: Long Form Question Answering
We introduce the first large-scale corpus for long-form question answering, a task requiring elaborate and in-depth answers to open-ended questions.
GLOSS: Generative Latent Optimization of Sentence Representations
Both are trained to reconstruct the sentence based on a latent code and our model can be used to generate text.
Generative Question Answering: Learning to Answer the Whole Question
Discriminative question answering models can overfit to superficial biases in datasets, because their loss function saturates when any clue makes the answer likely.
fairseq: A Fast, Extensible Toolkit for Sequence Modeling
fairseq is an open-source sequence modeling toolkit that allows researchers and developers to train custom models for translation, summarization, language modeling, and other text generation tasks.
Learning to Speak and Act in a Fantasy Text Adventure Game
We analyze the ingredients necessary for successful grounding in this setting, and how each of these factors relate to agents that can talk and act successfully.
Strategies for Structuring Story Generation
Writers generally rely on plans or sketches to write long stories, but most current language models generate word by word from left to right.
Pay Less Attention with Lightweight and Dynamic Convolutions
We predict separate convolution kernels based solely on the current time-step in order to determine the importance of context elements.
Ranked #1 on
Machine Translation
on WMT 2017 English-Chinese
Wizard of Wikipedia: Knowledge-Powered Conversational agents
In open-domain dialogue intelligent agents should exhibit the use of knowledge, however there are few convincing demonstrations of this to date.
Hierarchical Neural Story Generation
We explore story generation: creative systems that can build coherent and fluent passages of text about a topic.
Controllable Abstractive Summarization
Current models for document summarization disregard user preferences such as the desired length, style, the entities that the user might be interested in, or how much of the document the user has already read.
Prior matters: simple and general methods for evaluating and improving topic quality in topic modeling
In this work, we first show how the standard topic quality measures of coherence and pointwise mutual information act counter-intuitively in the presence of common but irrelevant words, making it difficult to even quantitatively identify situations in which topics may be dominated by stopwords.
Language Modeling with Gated Convolutional Networks
The pre-dominant approach to language modeling to date is based on recurrent neural networks.
Ranked #18 on
Language Modelling
on One Billion Word
