Search Results for author:
Found 56 papers, 22 papers with code
CapWAP: Image Captioning with a Purpose
In this task, we use question-answer (QA) pairs{---}a natural expression of information need{---}from users, instead of reference captions, for both training and post-inference evaluation.
Retrieval Augmented Language Model Pre-Training
Language model pre-training has been shown to capture a surprising amount of world knowledge, crucial for NLP tasks such as question answering.
Gecko: Versatile Text Embeddings Distilled from Large Language Models
On the Massive Text Embedding Benchmark (MTEB), Gecko with 256 embedding dimensions outperforms all existing entries with 768 embedding size.
MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructions
Image retrieval, i. e., finding desired images given a reference image, inherently encompasses rich, multi-faceted search intents that are difficult to capture solely using image-based measures.
Instruct-Imagen: Image Generation with Multi-modal Instruction
We introduce *multi-modal instruction* for image generation, a task representation articulating a range of generation intents with precision.
Gemini: A Family of Highly Capable Multimodal Models
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding.
Ranked #1 on
Multi-task Language Understanding
on MMLU
(using extra training data)
QUEST: A Retrieval Dataset of Entity-Seeking Queries with Implicit Set Operations
To study the ability of retrieval systems to meet such information needs, we construct QUEST, a dataset of 3357 natural language queries with implicit set operations, that map to a set of entities corresponding to Wikipedia documents.
Rethinking the Role of Token Retrieval in Multi-Vector Retrieval
Multi-vector retrieval models such as ColBERT [Khattab and Zaharia, 2020] allow token-level interactions between queries and documents, and hence achieve state of the art on many information retrieval benchmarks.
Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?
In this study, we introduce InfoSeek, a visual question answering dataset tailored for information-seeking questions that cannot be answered with only common sense knowledge.
Ranked #3 on
Visual Question Answering (VQA)
on InfoSeek
Open-domain Visual Entity Recognition: Towards Recognizing Millions of Wikipedia Entities
Large-scale multi-modal pre-training models such as CLIP and PaLI exhibit strong generalization on various visual domains and tasks.
Ranked #2 on
Fine-Grained Image Recognition
on OVEN
Meta-Learning Fast Weight Language Models
Dynamic evaluation of language models (LMs) adapts model parameters at test time using gradient information from previous tokens and substantially improves LM performance.
Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding
Visually-situated language is ubiquitous -- sources range from textbooks with diagrams to web pages with images and tables, to mobile apps with buttons and forms.
Ranked #17 on
Visual Question Answering (VQA)
on InfographicVQA
Promptagator: Few-shot Dense Retrieval From 8 Examples
To amplify the power of a few examples, we propose Prompt-base Query Generation for Retriever (Promptagator), which leverages large language models (LLM) as a few-shot query generator, and creates task-specific retrievers based on the generated data.
ASQA: Factoid Questions Meet Long-Form Answers
In contrast to existing long-form QA tasks (such as ELI5), ASQA admits a clear notion of correctness: a user faced with a good summary should be able to answer different interpretations of the original ambiguous question.
FRUIT: Faithfully Reflecting Updated Information in Text
Textual knowledge bases such as Wikipedia require considerable effort to keep up to date and consistent.
Large Dual Encoders Are Generalizable Retrievers
With multi-stage training, surprisingly, scaling up the model size brings significant improvement on a variety of retrieval tasks, especially for out-of-domain generalization.
Ranked #9 on
Zero-shot Text Search
on BEIR
Revisiting the Primacy of English in Zero-shot Cross-lingual Transfer
Zero-shot cross-lingual transfer is emerging as a practical solution: pre-trained models later fine-tuned on one transfer language exhibit surprising performance when tested on many target languages.
Joint Passage Ranking for Diverse Multi-Answer Retrieval
We study multi-answer retrieval, an under-explored problem that requires retrieving passages to cover multiple distinct answers for a given question.
Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representations
Sequence-to-sequence (seq2seq) models are prevalent in semantic parsing, but have been found to struggle at out-of-distribution compositional generalization.
Ranked #3 on
Semantic Parsing
on CFQ
CapWAP: Captioning with a Purpose
In this task, we use question-answer (QA) pairs---a natural expression of information need---from users, instead of reference captions, for both training and post-inference evaluation.
Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?
This has motivated new specialized architectures with stronger compositional biases, but most of these approaches have only been evaluated on synthetically-generated datasets, which are not representative of natural language variation.
Open Question Answering over Tables and Text
In open question answering (QA), the answer to a question is produced by retrieving and then analyzing documents that might contain answers to the question.
Ranked #1 on
Question Answering
on OTT-QA
Exploring Unexplored Generalization Challenges for Cross-Database Semantic Parsing
We study the task of cross-database semantic parsing (XSP), where a system that maps natural language utterances to executable SQL queries is evaluated on databases unseen during training.
Probabilistic Assumptions Matter: Improved Models for Distantly-Supervised Document-Level Question Answering
We address the problem of extractive question answering using document-level distant super-vision, pairing questions and relevant documents with answer strings.
REALM: Retrieval-Augmented Language Model Pre-Training
Language model pre-training has been shown to capture a surprising amount of world knowledge, crucial for NLP tasks such as question answering.
Ranked #9 on
Question Answering
on WebQuestions
Well-Read Students Learn Better: On the Importance of Pre-training Compact Models
Recent developments in natural language representations have been accompanied by large and expensive models that leverage vast amounts of general-domain text through self-supervised pre-training.
Zero-Shot Entity Linking by Reading Entity Descriptions
First, we show that strong reading comprehension models pre-trained on large unlabeled data can be used to generalize to unseen entities.
Handling Divergent Reference Texts when Evaluating Table-to-Text Generation
Automatically constructed datasets for generating text from semi-structured data (tables), such as WikiBio, often contain reference texts that diverge from the information in the corresponding semi-structured data.
Natural Questions: a Benchmark for Question Answering Research
The public release consists of 307, 373 training examples with single annotations, 7, 830 examples with 5-way annotations for development data, and a further 7, 842 examples 5-way annotated sequestered as test data.
Ranked #7 on
Question Answering
on Natural Questions (long)
Latent Retrieval for Weakly Supervised Open Domain Question Answering
We show for the first time that it is possible to jointly learn the retriever and reader from question-answer string pairs and without any IR system.
Ranked #10 on
Question Answering
on WebQuestions
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
In this paper we study yes/no questions that are naturally occurring --- meaning that they are generated in unprompted and unconstrained settings.
Ranked #26 on
Question Answering
on BoolQ
Language Model Pre-training for Hierarchical Document Representations
Hierarchical neural architectures are often used to capture long-distance dependencies and have been applied to many document-level tasks such as summarization, document segmentation, and sentiment analysis.
Improving Span-based Question Answering Systems with Coarsely Labeled Data
We study approaches to improve fine-grained short answer Question Answering models by integrating coarse-grained data annotated for paragraph-level relevance and show that coarsely annotated data can bring significant performance gains.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers.
Ranked #1 on
Named Entity Recognition
on CoNLL 2003 (English)
Policy Shaping and Generalized Update Equations for Semantic Parsing from Denotations
Semantic parsing from denotations faces two key challenges in model training: (1) given only the denotations (e. g., answers), search for good candidate semantic parses, and (2) choose the best model update algorithm.
Maximum Margin Reward Networks for Learning from Explicit and Implicit Supervision
Neural networks have achieved state-of-the-art performance on several structured-output prediction tasks, trained in a fully supervised fashion.
Modeling Large-Scale Structured Relationships with Shared Memory for Knowledge Base Completion
However, due to the size of knowledge bases, learning multi-step relations directly on top of observed triplets could be costly.
Search-based Neural Structured Learning for Sequential Question Answering
Recent work in semantic parsing for question answering has focused on long and complicated questions, many of which would seem unnatural if asked in a normal conversation between two humans.
A Knowledge-Grounded Neural Conversation Model
We generalize the widely-used Seq2Seq approach by conditioning responses on both conversation history and external "facts", allowing the model to be versatile and applicable in an open-domain setting.
From Entity Linking to Question Answering -- Recent Progress on Semantic Grounding Tasks
Entity linking and semantic parsing have been shown to be crucial to important applications such as question answering and document understanding.
Link Prediction using Embedded Knowledge Graphs
Since large knowledge bases are typically incomplete, missing facts need to be inferred from observed facts in a task called knowledge base completion.
Answering Complicated Question Intents Expressed in Decomposed Question Sequences
Recent work in semantic parsing for question answering has focused on long and complicated questions, many of which would seem unnatural if asked in a normal conversation between two humans.
Toward Socially-Infused Information Extraction: Embedding Authors, Mentions, and Entities
Entity linking is the task of identifying mentions of entities in text, and linking them to entries in a knowledge base.
S-MART: Novel Tree-based Structured Learning Algorithms Applied to Tweet Entity Linking
Non-linear models recently receive a lot of attention as people are starting to discover the power of statistical and embedding features.
Annotating Derivations: A New Evaluation Strategy and Dataset for Algebra Word Problems
We propose a new evaluation for automatic solvers for algebra word problems, which can identify mistakes that existing evaluations overlook.
IllinoisSL: A JAVA Library for Structured Prediction
IllinoisSL is a Java library for learning structured prediction models.
Inferring Missing Entity Type Instances for Knowledge Base Completion: New Dataset and Methods
In this work, we focus on the task of inferring missing entity type instances in a KB, a fundamental task for KB competition yet receives little attention.
Entity Linking on Microblogs with Spatial and Temporal Signals
Microblogs present an excellent opportunity for monitoring and analyzing world happenings.
Dual Coordinate Descent Algorithms for Efficient Large Margin Structured Prediction
Due to the nature of complex NLP problems, structured prediction algorithms have been important modeling tools for a wide range of tasks.
