Search Results for author:
Found 35 papers, 23 papers with code
Out of Style: RAG's Fragility to Linguistic Variation
Despite the impressive performance of Retrieval-augmented Generation (RAG) systems across various NLP benchmarks, their robustness in handling real-world user-LLM interaction queries remains largely underexplored.
OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs
Scientific progress depends on researchers' ability to synthesize the growing body of literature.
Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languages
Despite recent advances in multimodal large language models (MLLMs), their development has predominantly focused on English- and western-centric datasets and tasks, leaving most of the world's languages and diverse cultural contexts underrepresented.
Scaling Retrieval-Based Language Models with a Trillion-Token Datastore
Specifically, we find that increasing the size of the datastore used by a retrieval-based LM monotonically improves language modeling and several downstream tasks without obvious saturation, such that a smaller model augmented with a large datastore outperforms a larger LM-only model on knowledge-intensive tasks.
CopyBench: Measuring Literal and Non-Literal Reproduction of Copyright-Protected Text in Language Model Generation
Evaluating the degree of reproduction of copyright-protected content by language models (LMs) is of significant interest to the AI and legal communities.
CodeRAG-Bench: Can Retrieval Augment Code Generation?
We aggregate documents from five sources for models to retrieve contexts: competition solutions, online tutorials, library documentation, StackOverflow posts, and GitHub repositories.
Reliable, Adaptable, and Attributable Language Models with Retrieval
Parametric language models (LMs), which are trained on vast amounts of web data, exhibit remarkable flexibility and capability.
Fine-grained Hallucination Detection and Editing for Language Models
On our benchmark, our automatic and human evaluations show that FAVA significantly outperforms ChatGPT and GPT-4 on fine-grained hallucination detection, and edits suggested by FAVA improve the factuality of LM-generated text.
Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
Our framework trains a single arbitrary LM that adaptively retrieves passages on-demand, and generates and reflects on retrieved passages and its own generations using special tokens, called reflection tokens.
Ranked #1 on
Question Answering
on PopQA
BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfer
Despite remarkable advancements in few-shot generalization in natural language processing, most models are developed and evaluated primarily in English.
TaskWeb: Selecting Better Source Tasks for Multi-task NLP
TaskShop uses TaskWeb to estimate the benefit of using a source task for learning a new target task, and to choose a subset of helpful training tasks for multi-task training.
xPQA: Cross-Lingual Product Question Answering across 12 Languages
To study this practical industrial task, we present xPQA, a large-scale annotated cross-lingual PQA dataset in 12 languages across 9 branches, and report results in (1) candidate ranking, to select the best English candidate containing the information to answer a non-English question; and (2) answer generation, to generate a natural-sounding non-English answer based on the selected English candidate.
AfriQA: Cross-lingual Open-Retrieval Question Answering for African Languages
African languages have far less in-language content available digitally, making it challenging for question answering systems to satisfy the information needs of users.
How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrieval
We hence propose a new DA approach with diverse queries and sources of supervision to progressively train a generalizable DR. As a result, DRAGON, our dense retriever trained with diverse augmentation, is the first BERT-base-sized DR to achieve state-of-the-art effectiveness in both supervised and zero-shot evaluations and even competes with models using more complex late interaction (ColBERTv2 and SPLADE++).
Ranked #2 on
Passage Retrieval
on PeerQA
When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories
Despite their impressive performance on diverse tasks, large language models (LMs) still struggle with tasks requiring rich world knowledge, implying the limitations of relying solely on their parameters to encode a wealth of world knowledge.
Beyond Counting Datasets: A Survey of Multilingual Dataset Construction and Necessary Resources
While the NLP community is generally aware of resource disparities among languages, we lack research that quantifies the extent and types of such disparity.
Task-aware Retrieval with Instructions
We study the problem of retrieval with instructions, where users of a retrieval system explicitly describe their intent along with their queries.
RealTime QA: What's the Answer Right Now?
We introduce REALTIME QA, a dynamic question answering (QA) platform that announces questions and evaluates systems on a regular basis (weekly in this version).
MIA 2022 Shared Task: Evaluating Cross-lingual Open-Retrieval Question Answering for 16 Diverse Languages
We present the results of the Workshop on Multilingual Information Access (MIA) 2022 Shared Task, evaluating cross-lingual open-retrieval question answering (QA) systems in 16 typologically diverse languages.
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts
Our method, called ATTEMPT (ATTEntional Mixtures of Prompt Tuning), obtains source prompts as encodings of large-scale source tasks into a small number of parameters and trains an attention module to interpolate the source prompts and a newly initialized target prompt for every instance in the target task.
Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks
We introduce a multi-task learning framework to jointly generate the final output and predict the evidentiality of each passage, leveraging a new task-agnostic method to obtain silver evidentiality labels for supervision.
One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval
We present Cross-lingual Open-Retrieval Answer Generation (CORA), the first unified many-to-many question answering (QA) model that can answer questions across many languages, even for ones without language-specific annotated data or knowledge sources.
Efficient Passage Retrieval with Hashing for Open-domain Question Answering
Most state-of-the-art open-domain question answering systems use a neural retrieval model to encode passages into continuous vectors and extract them from a knowledge source.
Ranked #2 on
Open-Domain Question Answering
on TQA
MultiModalQA: Complex Question Answering over Text, Tables and Images
When answering complex questions, people can seamlessly combine information from visual, textual and tabular sources.
The Aleatoric Uncertainty Estimation Using a Separate Formulation with Virtual Residuals
We propose a new optimization framework for aleatoric uncertainty estimation in regression problems.
XOR QA: Cross-lingual Open-Retrieval Question Answering
Multilingual question answering tasks typically assume answers exist in the same language as the question.
Challenges in Information-Seeking QA: Unanswerable Questions and Paragraph Retrieval
However, datasets containing information-seeking queries where evidence documents are provided after the queries are written independently remain challenging.
LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention
In this paper, we propose new pretrained contextualized representations of words and entities based on the bidirectional transformer.
Ranked #1 on
Entity Typing
on Open Entity
Logic-Guided Data Augmentation and Regularization for Consistent Question Answering
Many natural language questions require qualitative, quantitative or logical comparisons between two entities or events.
Inferential Text Generation with Multiple Knowledge Sources and Meta-Learning
In this work, we use multiple knowledge sources as fuels for the model.
Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERT
There is an increasing amount of literature that claims the brittleness of deep neural networks in dealing with adversarial examples that are created maliciously.
Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering
Answering questions that require multi-hop reasoning at web-scale necessitates retrieving multiple evidence documents, one of which often has little lexical or semantic relationship to the question.
Ranked #26 on
Question Answering
on HotpotQA
Wikipedia2Vec: An Efficient Toolkit for Learning and Visualizing the Embeddings of Words and Entities from Wikipedia
The embeddings of entities in a large knowledge base (e. g., Wikipedia) are highly beneficial for solving various natural language tasks that involve real world knowledge.
Multilingual Extractive Reading Comprehension by Runtime Machine Translation
Given a target language without RC training data and a pivot language with RC training data (e. g. English), our method leverages existing RC resources in the pivot language by combining a competitive RC model in the pivot language with an attentive Neural Machine Translation (NMT) model.
HappyDB: A Corpus of 100,000 Crowdsourced Happy Moments
The science of happiness is an area of positive psychology concerned with understanding what behaviors make people happy in a sustainable fashion.
