Search Results for author:
Found 13 papers, 7 papers with code
CM3: A Causal Masked Multimodal Model of the Internet
We introduce CM3, a family of causally masked generative models trained over a large corpus of structured multi-modal documents that can contain both text and image tokens.
The Web Is Your Oyster -- Knowledge-Intensive NLP against a Very Large Web Corpus
In order to address increasing demands of real-world applications, the research for knowledge-intensive NLP (KI-NLP) should advance by capturing the challenges of a truly open-domain environment: web-scale knowledge, lack of structure, inconsistent quality and noise.
Discourse-Aware Soft Prompting for Text Generation
We show that soft-prompt based conditional text generation can be improved with simple and efficient methods that simulate modeling the discourse structure of human written text.
Domain-matched Pre-training Tasks for Dense Retrieval
Pre-training on larger datasets with ever increasing model size is now a proven recipe for increased performance across almost all NLP tasks.
Ranked #2 on
Passage Retrieval
on Natural Questions
(using extra training data)
NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
We review the EfficientQA competition from NeurIPS 2020.
Multi-task Retrieval for Knowledge-Intensive Tasks
Retrieving relevant contexts from a large corpus is a crucial step for tasks such as open-domain question answering and fact checking.
Joint Verification and Reranking for Open Fact Checking Over Tables
Structured information is an important knowledge source for automatic verification of factual claims.
UniK-QA: Unified Representations of Structured and Unstructured Knowledge for Open-Domain Question Answering
We study open-domain question answering with structured, unstructured and semi-structured knowledge sources, including text, tables, lists and knowledge bases.
Ranked #1 on
Open-Domain Question Answering
on WebQuestions
(using extra training data)
Knowledge Base Question Answering
Open-Domain Question Answering
KILT: a Benchmark for Knowledge Intensive Language Tasks
We test both task-specific and general baselines, evaluating downstream performance in addition to the ability of the models to provide provenance.
Ranked #3 on
Entity Linking
on KILT: WNED-CWEB
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Large pre-trained language models have been shown to store factual knowledge in their parameters, and achieve state-of-the-art results when fine-tuned on downstream NLP tasks.
Ranked #4 on
Question Answering
on WebQuestions
Dense Passage Retrieval for Open-Domain Question Answering
Open-domain question answering relies on efficient passage retrieval to select candidate contexts, where traditional sparse vector space models, such as TF-IDF or BM25, are the de facto method.
Ranked #1 on
Question Answering
on NaturalQA
Aligned Cross Entropy for Non-Autoregressive Machine Translation
This difficultly is compounded during training with cross entropy loss, which can highly penalize small shifts in word order.
Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translation
We consider the problem of making machine translation more robust to character-level variation at the source side, such as typos.
