Search Results for author:
Found 13 papers, 11 papers with code
Making Retrieval-Augmented Language Models Robust to Irrelevant Context
An important desideratum of RALMs, is that retrieved information helps model performance when it is relevant, and does not harm performance when it is not.
Ranked #3 on
Question Answering
on Bamboogle
Generating Benchmarks for Factuality Evaluation of Language Models
FACTOR automatically transforms a factual corpus of interest into a benchmark evaluating an LM's propensity to generate true facts from the corpus vs. similar but incorrect statements.
In-Context Retrieval-Augmented Language Models
Retrieval-Augmented Language Modeling (RALM) methods, which condition a language model (LM) on relevant documents from a grounding corpus during generation, were shown to significantly improve language modeling performance.
Parallel Context Windows for Large Language Models
We present Parallel Context Windows (PCW), a method that alleviates the context window restriction for any off-the-shelf LLM without further training.
What Are You Token About? Dense Retrieval as Distributions Over the Vocabulary
We leverage this insight and propose a simple way to enrich query and passage representations with lexical information at inference time, and show that this significantly improves performance compared to the original model in zero-shot settings, and specifically on the BEIR benchmark.
Standing on the Shoulders of Giant Frozen Language Models
To demonstrate this, we introduce three novel methods for leveraging frozen models: input-dependent prompt tuning, frozen readers, and recursive LMs, each of which vastly improves on current frozen-model approaches.
Transformer Language Models without Positional Encodings Still Learn Positional Information
Causal transformer language models (LMs), such as GPT-3, typically require some form of positional encoding, such as positional embeddings.
Learning to Retrieve Passages without Supervision
Dense retrievers for open-domain question answering (ODQA) have been shown to achieve impressive performance by training on large datasets of question-passage pairs.
How Optimal is Greedy Decoding for Extractive Question Answering?
However, this approach does not ensure that the answer is a span in the given passage, nor does it guarantee that it is the most probable one.
Few-Shot Question Answering by Pretraining Span Selection
Given a passage with multiple sets of recurring spans, we mask in each set all recurring spans but one, and ask the model to select the correct span in the passage for each masked span.
Coreference Resolution without Span Representations
The introduction of pretrained language models has reduced many complex task-specific NLP models to simple lightweight layers.
Ranked #5 on
Coreference Resolution
on CoNLL 2012
SenseBERT: Driving Some Sense into BERT
The ability to learn from large unlabeled corpora has allowed neural language models to advance the frontier in natural language understanding.
Ranked #11 on
Word Sense Disambiguation
on Words in Context
Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsing
We introduce a novel method for multilingual transfer that utilizes deep contextual embeddings, pretrained in an unsupervised fashion.
Cross-lingual zero-shot dependency parsing
Few-Shot Learning
+1
