Search Results for author:
Found 15 papers, 9 papers with code
CommonsenseQA 2.0: Exposing the Limits of AI through Gamification
Constructing benchmarks that test the abilities of modern natural language understanding models is difficult - pre-trained language models exploit artifacts in benchmarks to achieve human parity, but still fail on adversarial examples and make errors that demonstrate a lack of common sense.
Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills
Models pre-trained with a language modeling objective possess ample world knowledge and language skills, but are known to struggle in tasks that require reasoning.
MultiModalQA: Complex Question Answering over Text, Tables and Images
When answering complex questions, people can seamlessly combine information from visual, textual and tabular sources.
Leap-Of-Thought: Teaching Pre-Trained Models to Systematically Reason Over Implicit Knowledge
In this work, we provide a first demonstration that LMs can be trained to reliably perform systematic reasoning combining both implicit, pre-trained knowledge and explicit natural language statements.
oLMpics -- On what Language Model Pre-training Captures
A fundamental challenge is to understand whether the performance of a LM on a task should be attributed to the pre-trained representations or to the process of fine-tuning on the task data.
ORB: An Open Reading Benchmark for Comprehensive Evaluation of Machine Reading Comprehension
A lot of diverse reading comprehension datasets have recently been introduced to study various phenomena in natural language, ranging from simple paraphrase matching and entity typing to entity tracking and understanding the implications of the context.
On Making Reading Comprehension More Comprehensive
In this work, we justify a question answering approach to reading comprehension and describe the various kinds of questions one might use to more fully test a system{'}s comprehension of a passage, moving beyond questions that only probe local predicate-argument structures.
Comprehensive Multi-Dataset Evaluation of Reading Comprehension
A lot of diverse reading comprehension datasets have recently been introduced to study various phenomena in natural language, ranging from simple paraphrase matching and entity typing to entity tracking and understanding the implications of the context.
MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension
We present the results of the Machine Reading for Question Answering (MRQA) 2019 shared task on evaluating the generalization capabilities of reading comprehension systems.
Question Answering is a Format; When is it Useful?
In this opinion piece, we argue that question answering should be considered a format which is sometimes useful for studying particular phenomena, not a phenomenon or task in itself.
MultiQA: An Empirical Investigation of Generalization and Transfer in Reading Comprehension
A large number of reading comprehension (RC) datasets has been created recently, but little analysis has been done on whether they generalize to one another, and the extent to which existing datasets can be leveraged for improving performance on new ones.
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
To investigate question answering with prior knowledge, we present CommonsenseQA: a challenging new dataset for commonsense question answering.
Ranked #31 on
Common Sense Reasoning
on CommonsenseQA
(using extra training data)
Repartitioning of the ComplexWebQuestions Dataset
Recently, Talmor and Berant (2018) introduced ComplexWebQuestions - a dataset focused on answering complex questions by decomposing them into a sequence of simpler questions and extracting the answer from retrieved web snippets.
The Web as a Knowledge-base for Answering Complex Questions
In this paper, we present a novel framework for answering broad and complex questions, assuming answering simple questions is possible using a search engine and a reading comprehension model.
Evaluating Semantic Parsing against a Simple Web-based Question Answering Model
Semantic parsing shines at analyzing complex natural language that involves composition and computation over multiple pieces of evidence.
Ranked #1 on
Question Answering
on COMPLEXQUESTIONS
