Search Results for author:
Found 46 papers, 18 papers with code
Understanding the World's Museums through Vision-Language Reasoning
In this work, we facilitate such reasoning by (a) collecting and curating a large-scale dataset of 65M images and 200M question-answer pairs in the standard museum catalog format for exhibits from all around the world; (b) training large vision-language models on the collected dataset; (c) benchmarking their ability on five visual question answering tasks.
ALTA: Compiler-Based Analysis of Transformers
We propose a new programming language called ALTA and a compiler that can map ALTA programs to Transformer weights.
Taming CLIP for Fine-grained and Structured Visual Understanding of Museum Exhibits
In this work, we aim to adapt CLIP for fine-grained and structured -- in the form of tabular data -- visual understanding of museum exhibits.
Mitigating Catastrophic Forgetting in Language Transfer via Model Merging
As open-weight large language models (LLMs) achieve ever more impressive performances across a wide range of tasks in English, practitioners aim to adapt these models to different languages.
Efficient End-to-End Visual Document Understanding with Rationale Distillation
Pre-processing tools, such as optical character recognition (OCR), can map document image inputs to textual tokens, then large language models (LLMs) can reason over text.
From Pixels to UI Actions: Learning to Follow Instructions via Graphical User Interfaces
Much of the previous work towards digital agents for graphical user interfaces (GUIs) has relied on text-based representations (derived from HTML or other structured data sources), which are not always readily available.
Anchor Prediction: Automatic Refinement of Internet Links
Internet links enable users to deepen their understanding of a topic by providing convenient access to related information.
QUEST: A Retrieval Dataset of Entity-Seeking Queries with Implicit Set Operations
To study the ability of retrieval systems to meet such information needs, we construct QUEST, a dataset of 3357 natural language queries with implicit set operations, that map to a set of entities corresponding to Wikipedia documents.
Open-domain Visual Entity Recognition: Towards Recognizing Millions of Wikipedia Entities
Large-scale multi-modal pre-training models such as CLIP and PaLI exhibit strong generalization on various visual domains and tasks.
Ranked #2 on
Fine-Grained Image Recognition
on OVEN
Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding
Visually-situated language is ubiquitous -- sources range from textbooks with diagrams to web pages with images and tables, to mobile apps with buttons and forms.
Ranked #18 on
Visual Question Answering (VQA)
on InfographicVQA
Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsing
Meanwhile, recent work has shown considerable improvements on many NLP tasks from model scaling.
Improving Compositional Generalization with Latent Structure and Data Augmentation
Generic unstructured neural networks have been shown to struggle on out-of-distribution compositional generalization.
Revisiting the Primacy of English in Zero-shot Cross-lingual Transfer
Zero-shot cross-lingual transfer is emerging as a practical solution: pre-trained models later fine-tuned on one transfer language exhibit surprising performance when tested on many target languages.
Joint Passage Ranking for Diverse Multi-Answer Retrieval
We study multi-answer retrieval, an under-explored problem that requires retrieving passages to cover multiple distinct answers for a given question.
Representations for Question Answering from Documents with Tables and Text
Tables in Web documents are pervasive and can be directly used to answer many of the queries searched on the Web, motivating their integration in question answering.
Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?
This has motivated new specialized architectures with stronger compositional biases, but most of these approaches have only been evaluated on synthetically-generated datasets, which are not representative of natural language variation.
Probabilistic Assumptions Matter: Improved Models for Distantly-Supervised Document-Level Question Answering
We address the problem of extractive question answering using document-level distant super-vision, pairing questions and relevant documents with answer strings.
Sparse, Dense, and Attentional Representations for Text Retrieval
Dual encoders perform retrieval by encoding documents and queries into dense lowdimensional vectors, scoring each document by its inner product with the query.
Contextualized Representations Using Textual Encyclopedic Knowledge
We present a method to represent input texts by contextualizing them jointly with dynamically retrieved textual encyclopedic background knowledge from multiple documents.
Well-Read Students Learn Better: On the Importance of Pre-training Compact Models
Recent developments in natural language representations have been accompanied by large and expensive models that leverage vast amounts of general-domain text through self-supervised pre-training.
Zero-Shot Entity Linking by Reading Entity Descriptions
First, we show that strong reading comprehension models pre-trained on large unlabeled data can be used to generalize to unseen entities.
Latent Retrieval for Weakly Supervised Open Domain Question Answering
We show for the first time that it is possible to jointly learn the retriever and reader from question-answer string pairs and without any IR system.
Ranked #11 on
Question Answering
on WebQuestions
Natural Questions: a Benchmark for Question Answering Research
The public release consists of 307, 373 training examples with single annotations, 7, 830 examples with 5-way annotations for development data, and a further 7, 842 examples 5-way annotated sequestered as test data.
Ranked #7 on
Question Answering
on Natural Questions (long)
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
In this paper we study yes/no questions that are naturally occurring --- meaning that they are generated in unprompted and unconstrained settings.
Ranked #27 on
Question Answering
on BoolQ
Language Model Pre-training for Hierarchical Document Representations
Hierarchical neural architectures are often used to capture long-distance dependencies and have been applied to many document-level tasks such as summarization, document segmentation, and sentiment analysis.
Improving Span-based Question Answering Systems with Coarsely Labeled Data
We study approaches to improve fine-grained short answer Question Answering models by integrating coarse-grained data annotated for paragraph-level relevance and show that coarsely annotated data can bring significant performance gains.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers.
Ranked #1 on
Named Entity Recognition
on SciERC
Cross-Sentence N-ary Relation Extraction with Graph LSTMs
Past work in relation extraction has focused on binary relations in single sentences.
A Nested Attention Neural Hybrid Model for Grammatical Error Correction
Grammatical error correction (GEC) systems strive to correct both global errors in word order and usage, and local errors in spelling and inflection.
NLP for Precision Medicine
We will introduce precision medicine and showcase the vast opportunities for NLP in this burgeoning field with great societal impact.
E-TIPSY: Search Query Corpus Annotated with Entities, Term Importance, POS Tags, and Syntactic Parses
In addition, it contains automatically produced annotations of named entities, part-of-speech tags, and syntactic parses for the same queries.
