Search Results for author:
Found 13 papers, 4 papers with code
MEMORY-VQ: Compression for Tractable Internet-Scale Memory
Retrieval augmentation is a powerful but expensive method to make language models more knowledgeable about the world.
GLIMMER: generalized late-interaction memory reranker
Memory-augmentation is a powerful approach for efficiently incorporating external information into language models, but leads to reduced performance relative to retrieving text.
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
Multi-query attention (MQA), which only uses a single key-value head, drastically speeds up decoder inference.
CoLT5: Faster Long-Range Transformers with Conditional Computation
Many natural language processing tasks benefit from long inputs, but processing long documents with Transformers is expensive -- not only due to quadratic attention complexity but also from applying feedforward and projection layers to every token.
Ranked #1 on
Long-range modeling
on SCROLLS
Pre-computed memory or on-the-fly encoding? A hybrid approach to retrieval augmentation makes the most of your compute
Retrieval-augmented language models such as Fusion-in-Decoder are powerful, setting the state of the art on a variety of knowledge-intensive tasks.
FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference
Fusion-in-Decoder (FiD) is a powerful retrieval-augmented language model that sets the state-of-the-art on many knowledge-intensive NLP tasks.
Ranked #4 on
Question Answering
on WebQuestions
Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models
Decoding methods for large language models often trade-off between diversity of outputs and parallelism of computation.
Generate-and-Retrieve: use your predictions to improve retrieval for semantic parsing
Then, it retrieves exemplars with outputs similar to the preliminary prediction which are used to generate a final prediction.
Mention Memory: incorporating textual knowledge into Transformers through entity mention attention
We propose to address this problem by integrating a semi-parametric representation of a large text corpus into a Transformer model as a source of factual knowledge.
Ranked #1 on
Passage Retrieval
on EntityQuestions
ReadTwice: Reading Very Large Documents with Memories
Knowledge-intensive tasks such as question answering often require assimilating information from different sections of large inputs such as books or article collections.
DOCENT: Learning Self-Supervised Entity Representations from Large Document Collections
This enables a new class of powerful, high-capacity representations that can ultimately distill much of the useful information about an entity from multiple text sources, without any human supervision.
Self-Attentive, Multi-Context One-Class Classification for Unsupervised Anomaly Detection on Text
There exist few text-specific methods for unsupervised anomaly detection, and for those that do exist, none utilize pre-trained models for distributed vector representations of words.
Aiming to Know You Better Perhaps Makes Me a More Engaging Dialogue Partner
There have been several attempts to define a plausible motivation for a chit-chat dialogue agent that can lead to engaging conversations.
