Search Results for author:
Found 14 papers, 3 papers with code
MEMORY-VQ: Compression for Tractable Internet-Scale Memory
Retrieval augmentation is a powerful but expensive method to make language models more knowledgeable about the world.
GLIMMER: generalized late-interaction memory reranker
Memory-augmentation is a powerful approach for efficiently incorporating external information into language models, but leads to reduced performance relative to retrieving text.
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
Multi-query attention (MQA), which only uses a single key-value head, drastically speeds up decoder inference.
CoLT5: Faster Long-Range Transformers with Conditional Computation
Many natural language processing tasks benefit from long inputs, but processing long documents with Transformers is expensive -- not only due to quadratic attention complexity but also from applying feedforward and projection layers to every token.
Ranked #1 on
Long-range modeling
on SCROLLS
Pre-computed memory or on-the-fly encoding? A hybrid approach to retrieval augmentation makes the most of your compute
Retrieval-augmented language models such as Fusion-in-Decoder are powerful, setting the state of the art on a variety of knowledge-intensive tasks.
FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference
Fusion-in-Decoder (FiD) is a powerful retrieval-augmented language model that sets the state-of-the-art on many knowledge-intensive NLP tasks.
Ranked #3 on
Question Answering
on WebQuestions
Generate-and-Retrieve: use your predictions to improve retrieval for semantic parsing
Then, it retrieves exemplars with outputs similar to the preliminary prediction which are used to generate a final prediction.
QA Is the New KR: Question-Answer Pairs as Knowledge Bases
In this position paper, we propose a new approach to generating a type of knowledge base (KB) from text, based on question generation and entity linking.
Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering
Retrieval augmented language models have recently become the standard for knowledge intensive tasks.
Mention Memory: incorporating textual knowledge into Transformers through entity mention attention
We propose to address this problem by integrating a semi-parametric representation of a large text corpus into a Transformer model as a source of factual knowledge.
Ranked #1 on
Passage Retrieval
on EntityQuestions
Grounding Complex Navigational Instructions Using Scene Graphs
Training a reinforcement learning agent to carry out natural language instructions is limited by the available supervision, i. e. knowing when the instruction has been carried out.
ReadTwice: Reading Very Large Documents with Memories
Knowledge-intensive tasks such as question answering often require assimilating information from different sections of large inputs such as books or article collections.
Neural Theorem Provers Do Not Learn Rules Without Exploration
Neural symbolic processing aims to combine the generalization of logical learning approaches and the performance of neural networks.
Weighted Global Normalization for Multiple Choice Reading Comprehension over Long Documents
Motivated by recent evidence pointing out the fragility of high-performing span prediction models, we direct our attention to multiple choice reading comprehension.
