Search Results for author:
Found 128 papers, 82 papers with code
Reframing Instructional Prompts to GPTk’s Language
Our experiments compare the zero-shot and few-shot performance of LMs prompted with reframed instructions on 12 NLP tasks across 6 categories.
Retrieval Data Augmentation Informed by Downstream Question Answering Performance
Training retrieval models to fetch contexts for Question Answering (QA) over large corpora requires labeling relevant passages in those corpora.
ART: Automatic multi-step reasoning and tool-use for large language models
We introduce Automatic Reasoning and Tool-use (ART), a framework that uses frozen LLMs to automatically generate intermediate reasoning steps as a program.
Do Embodied Agents Dream of Pixelated Sheep?: Embodied Decision Making using Language Guided World Modelling
Reinforcement learning (RL) agents typically learn tabula rasa, without prior knowledge of the world, which makes learning complex tasks with sparse rewards difficult.
Self-Instruct: Aligning Language Model with Self Generated Instructions
Applying our method to vanilla GPT3, we demonstrate a 33% absolute improvement over the original model on Super-NaturalInstructions, on par with the performance of InstructGPT_001, which is trained with private user data and human annotations.
HINT: Hypernetwork Instruction Tuning for Efficient Zero-Shot Generalisation
Recent NLP models have the great ability to generalise `zero-shot' to new tasks using only an instruction as guidance.
When Not to Trust Language Models: Investigating Effectiveness and Limitations of Parametric and Non-Parametric Memories
Despite their impressive performance on diverse tasks, large language models (LMs) still struggle with tasks requiring rich world knowledge, implying the limitations of relying solely on their parameters to encode a wealth of world knowledge.
Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations
Although large language models can be prompted for both zero- and few-shot learning, performance drops significantly when no demonstrations are available.
Editing Models with Task Arithmetic
Changing how pre-trained models behave -- e. g., improving their performance on a downstream task or mitigating biases learned during pre-training -- is a common practice when developing machine learning systems.
AGRO: Adversarial Discovery of Error-prone groups for Robust Optimization
We propose AGRO -- Adversarial Group discovery for Distributionally Robust Optimization -- an end-to-end approach that jointly identifies error-prone groups and improves accuracy on them.
Nonparametric Masked Language Modeling
Existing language models (LMs) predict tokens with a softmax over a finite vocabulary, which can make it difficult to predict rare tokens or phrases.
Data-Efficient Finetuning Using Cross-Task Nearest Neighbors
We assume access to a small number (250--1000) of unlabeled target task instances, select their nearest neighbors from a pool of multitask data, and use the retrieved data to train target task-specific models.
CREPE: Open-Domain Question Answering with False Presuppositions
We find that 25% of questions contain false presuppositions, and provide annotations for these presuppositions and their corrections.
Task-aware Retrieval with Instructions
We study the problem of retrieval with instructions, where users of a retrieval system explicitly describe their intent along with their queries.
SciFact-Open: Towards open-domain scientific claim verification
While research on scientific claim verification has led to the development of powerful systems that appear to approach human performance, these approaches have yet to be tested in a realistic setting against large corpora of scientific literature.
Exploring The Landscape of Distributional Robustness for Question Answering Models
We conduct a large empirical evaluation to investigate the landscape of distributional robustness in question answering.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Abstractive summarization models often generate inconsistent summaries containing factual errors or hallucinated content.
CORE: A Retrieve-then-Edit Framework for Counterfactual Data Generation
We present COunterfactual Generation via Retrieval and Editing (CORE), a retrieval-augmented generation framework for creating diverse counterfactual perturbations for CDA.
Rainier: Reinforced Knowledge Introspector for Commonsense Question Answering
Our work is the first to report that knowledge generated by models that are orders of magnitude smaller than GPT-3, even without direct supervision on the knowledge itself, can exceed the quality of commonsense knowledge elicited from GPT-3.
Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization
To help answer this, we first introduce an open-source modular library, RL4LMs (Reinforcement Learning for Language Models), for optimizing language generators with RL.
Patching open-vocabulary models by interpolating weights
We study model patching, where the goal is to improve accuracy on specific tasks without degrading accuracy on tasks where performance is already adequate.
INSCIT: Information-Seeking Conversations with Mixed-Initiative Interactions
In an information-seeking conversation, a user converses with an agent to ask a series of questions that can often be under- or over-specified.
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
NaturalProver: Grounded Mathematical Proof Generation with Language Models
Theorem proving in natural mathematical language - the mixture of symbolic and natural language used by humans - plays a central role in mathematical advances and education, and tests aspects of reasoning that are core to intelligence.
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts
Our method, called ATTEMPT (ATTEntional Mixtures of Prompt Tuning), obtains source prompts as encodings of large-scale source tasks into a small number of parameters and trains an attention module to interpolate the source prompts and a newly initialized target prompt for every instance in the target task.
Aligning to Social Norms and Values in Interactive Narratives
We focus on creating agents that act in alignment with socially beneficial norms and values in interactive narratives or text-based games -- environments wherein an agent perceives and interacts with a world through natural language.
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks
This large and diverse collection of tasks enables rigorous benchmarking of cross-task generalization under instructions -- training models to follow instructions on a subset of tasks and evaluating them on the remaining unseen ones.
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?
Large language models (LMs) are able to in-context learn -- perform a new task via inference alone by conditioning on a few input-label pairs (demonstrations) and making predictions for new inputs.
UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training
We present UnifiedQA-v2, a QA model built with the same process as UnifiedQA, except that it utilizes more supervision -- roughly 3x the number of datasets used for UnifiedQA.
Knowledge Base Question Answering by Case-based Reasoning over Subgraphs
Question answering (QA) over knowledge bases (KBs) is challenging because of the diverse, essentially unbounded, types of reasoning patterns needed.
CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learning
Compared to standard retrieval tasks, passage retrieval for conversational question answering (CQA) poses new challenges in understanding the current user question, as each question needs to be interpreted within the dialogue context.
Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks
We introduce a multi-task learning framework to jointly generate the final output and predict the evidentiality of each passage, leveraging a new task-agnostic method to obtain silver evidentiality labels for supervision.
Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts
Fine-tuning continuous prompts for target tasks has recently emerged as a compact alternative to full model fine-tuning.
MultiVerS: Improving scientific claim verification with weak supervision and full-document context
Our approach outperforms two competitive baselines on three scientific claim verification datasets, with particularly strong performance in zero / few-shot domain adaptation experiments.
Iconary: A Pictionary-Based Game for Testing Multimodal Communication with Drawings and Text
We investigate these challenges in the context of Iconary, a collaborative game of drawing and guessing based on Pictionary, that poses a novel challenge for the research community.
MetaICL: Learning to Learn In Context
We introduce MetaICL (Meta-training for In-Context Learning), a new meta-training framework for few-shot learning where a pretrained language model is tuned to do in-context learning on a large set of training tasks.
Generated Knowledge Prompting for Commonsense Reasoning
It remains an open question whether incorporating external knowledge benefits commonsense reasoning while maintaining the flexibility of pretrained sequence models.
Reframing Instructional Prompts to GPTk's Language
Our experiments compare the zero-shot and few-shot performance of LMs prompted with reframed instructions on 12 NLP tasks across 6 categories.
DIALKI: Knowledge Identification in Conversational Systems through Dialogue-Document Contextualization
Identifying relevant knowledge to be used in conversational systems that are grounded in long documents is critical to effective response generation.
Robust fine-tuning of zero-shot models
Compared to standard fine-tuning, WiSE-FT provides large accuracy improvements under distribution shift, while preserving high accuracy on the target distribution.
Ranked #8 on
Image Classification
on ObjectNet
(using extra training data)
Noisy Channel Language Model Prompting for Few-Shot Text Classification
We introduce a noisy channel approach for language model prompting in few-shot text classification.
One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval
We present Cross-lingual Open-Retrieval Answer Generation (CORA), the first unified many-to-many question answering (QA) model that can answer questions across many languages, even for ones without language-specific annotated data or knowledge sources.
FaVIQ: FAct Verification from Information-seeking Questions
Claims in FAVIQ are verified to be natural, contain little lexical bias, and require a complete understanding of the evidence for verification.
Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Study
Biomedical knowledge graphs (KGs) hold rich information on entities such as diseases, drugs, and genes.
Prompting Contrastive Explanations for Commonsense Reasoning Tasks
Many commonsense reasoning NLP tasks involve choosing between one or more possible answers to a question or prompt based on knowledge that is often implicit.
Efficient Passage Retrieval with Hashing for Open-domain Question Answering
Most state-of-the-art open-domain question answering systems use a neural retrieval model to encode passages into continuous vectors and extract them from a knowledge source.
Ranked #2 on
Open-Domain Question Answering
on TQA
Beyond Paragraphs: NLP for Long Sequences
In this tutorial, we aim at bringing interested NLP researchers up to speed about the recent and ongoing techniques for document-level representation learning.
GooAQ: Open Question Answering with Diverse Answer Types
GooAQ answers are mined from Google's responses to our collected questions, specifically from the answer boxes in the search results.
Cross-Task Generalization via Natural Language Crowdsourcing Instructions
Using this meta-dataset, we measure cross-task generalization by training models on seen tasks and measuring generalization to the remaining unseen ones.
Joint Passage Ranking for Diverse Multi-Answer Retrieval
We study multi-answer retrieval, an under-explored problem that requires retrieving passages to cover multiple distinct answers for a given question.
Probing Across Time: What Does RoBERTa Know and When?
Models of language trained on very large corpora have been demonstrated useful for NLP.
MultiModalQA: Complex Question Answering over Text, Tables and Images
When answering complex questions, people can seamlessly combine information from visual, textual and tabular sources.
NaturalProofs: Mathematical Theorem Proving in Natural Language
Understanding and creating mathematics using natural mathematical language - the mixture of symbolic and natural language used by humans - is a challenging and important problem for driving progress in machine learning.
NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
We review the EfficientQA competition from NeurIPS 2020.
IIRC: A Dataset of Incomplete Information Reading Comprehension Questions
However, most existing reading comprehension (RC) tasks only focus on questions for which the contexts provide all the information required to answer them, thus not evaluating a system's performance at identifying a potential lack of sufficient information and locating sources for that information.
XOR QA: Cross-lingual Open-Retrieval Question Answering
Multilingual question answering tasks typically assume answers exist in the same language as the question.
MedICaT: A Dataset of Medical Images, Captions, and Textual References
To address challenges in figure retrieval and figure-to-text alignment, we introduce MedICaT, a dataset of medical images in context.
Extracting a Knowledge Base of Mechanisms from COVID-19 Papers
The COVID-19 pandemic has spawned a diverse body of scientific literature that is challenging to navigate, stimulating interest in automated tools to help find useful knowledge.
X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformers
X-LXMERT's image generation capabilities rival state of the art generative models while its question answering and captioning abilities remains comparable to LXMERT.
Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics
Experiments across four datasets show that these model-dependent measures reveal three distinct regions in the data map, each with pronounced characteristics.
Extracting Summary Knowledge Graphs from Long Documents
Knowledge graphs capture entities and relations from long documents and can facilitate reasoning in many downstream applications.
DeLighT: Deep and Light-weight Transformer
We introduce a deep and light-weight transformer, DeLighT, that delivers similar or better performance than standard transformer-based models with significantly fewer parameters.
Ranked #1 on
Machine Translation
on WMT2016 English-Romanian
HATNet: An End-to-End Holistic Attention Network for Diagnosis of Breast Biopsy Images
HATNet extends the bag-of-words approach and uses self-attention to encode global information, allowing it to learn representations from clinically relevant tissue structures without any explicit supervision.
Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web
In this tutorial we take a holistic view toward information extraction, exploring the commonalities in the challenges and solutions developed to address these different forms of text.
ZeroShotCeres: Zero-Shot Relation Extraction from Semi-Structured Webpages
In this work, we propose a solution for "zero-shot" open-domain relation extraction from webpages with a previously unseen template, including from websites with little overlap with existing sources of knowledge for distant supervision and websites in entirely new subject verticals.
UnifiedQA: Crossing Format Boundaries With a Single QA System
As evidence, we use the latest advances in language modeling to build a single pre-trained QA model, UnifiedQA, that performs surprisingly well across 17 QA datasets spanning 4 diverse formats.
Ranked #1 on
Question Answering
on CommonsenseQA
SciREX: A Challenge Dataset for Document-Level Information Extraction
It is challenging to create a large-scale information extraction (IE) dataset at the document level since it requires an understanding of the whole document to annotate entities and their document-level relationships that usually span beyond sentences or even sections.
A Controllable Model of Grounded Response Generation
Current end-to-end neural conversation models inherently lack the flexibility to impose semantic control in the response generation process, often resulting in uninteresting responses.
Probing Contextual Language Models for Common Ground with Visual Representations
The success of large-scale contextual language models has attracted great interest in probing what is encoded in their representations.
An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction
Decisions of complex language understanding models can be rationalized by limiting their inputs to a relevant subsequence of the original text.
Fact or Fiction: Verifying Scientific Claims
We introduce scientific claim verification, a new task to select abstracts from the research literature containing evidence that SUPPORTS or REFUTES a given scientific claim, and to identify rationales justifying each decision.
AmbigQA: Answering Ambiguous Open-domain Questions
Ambiguity is inherent to open-domain question answering; especially when exploring new topics, it can be difficult to ask questions that have a single, unambiguous answer.
Logic-Guided Data Augmentation and Regularization for Consistent Question Answering
Many natural language questions require qualitative, quantitative or logical comparisons between two entities or events.
Procedural Reading Comprehension with Attribute-Aware Context Flow
Procedural texts often describe processes (e. g., photosynthesis and cooking) that happen over entities (e. g., light, food).
Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping
We publicly release all of our experimental data, including training and validation scores for 2, 100 trials, to encourage further analysis of training dynamics during fine-tuning.
DeFINE: DEep Factorized INput Token Embeddings for Neural Sequence Modeling
For sequence models with large vocabularies, a majority of network parameters lie in the input and output layers.
Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering
Answering questions that require multi-hop reasoning at web-scale necessitates retrieving multiple evidence documents, one of which often has little lexical or semantic relationship to the question.
Ranked #28 on
Question Answering
on HotpotQA
Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering
We introduce an approach for open-domain question answering (QA) that retrieves and reads a passage graph, where vertices are passages of text and edges represent relationships that are derived from an external knowledge base or co-occurrence in the same article.
Contextualized Sparse Representations for Real-Time Open-Domain Question Answering
Open-domain question answering can be formulated as a phrase retrieval problem, in which we can expect huge scalability and speed benefit but often suffer from low accuracy due to the limitation of existing phrase representation models.
On Making Reading Comprehension More Comprehensive
In this work, we justify a question answering approach to reading comprehension and describe the various kinds of questions one might use to more fully test a system{'}s comprehension of a passage, moving beyond questions that only probe local predicate-argument structures.
Question Answering is a Format; When is it Useful?
In this opinion piece, we argue that question answering should be considered a format which is sometimes useful for studying particular phenomena, not a phenomenon or task in itself.
A Discrete Hard EM Approach for Weakly Supervised Question Answering
Many question answering (QA) tasks only provide weak supervision for how the answer should be computed.
Ranked #2 on
Question Answering
on NarrativeQA
Entity, Relation, and Event Extraction with Contextualized Span Representations
We examine the capabilities of a unified, multi-task framework for three information extraction tasks: named entity recognition, relation extraction, and event extraction.
Ranked #5 on
Joint Entity and Relation Extraction
on SciERC
Mixture Content Selection for Diverse Sequence Generation
The diversification stage uses a mixture of experts to sample different binary masks on the source sequence for diverse content selection.
Ranked #10 on
Question Generation
on SQuAD1.1
Potential-Based Advice for Stochastic Policy Learning
This paper augments the reward received by a reinforcement learning agent with potential functions in order to help the agent learn (possibly stochastic) optimal policies.
Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index
Existing open-domain question answering (QA) models are not suitable for real-time usage because they need to process several long documents on-demand for every input query.
DiCENet: Dimension-wise Convolutions for Efficient Networks
When DiCE units are stacked to build the DiCENet model, we observe significant improvements over state-of-the-art models across various computer vision tasks including image classification, object detection, and semantic segmentation.
Ranked #20 on
Semantic Segmentation
on PASCAL VOC 2012 val
Multi-hop Reading Comprehension through Question Decomposition and Rescoring
Multi-hop Reading Comprehension (RC) requires reasoning and aggregation across several paragraphs.
Ranked #68 on
Question Answering
on HotpotQA
Compositional Questions Do Not Necessitate Multi-hop Reasoning
Multi-hop reading comprehension (RC) questions are challenging because they require reading and reasoning over multiple paragraphs.
SemEval-2019 Task 10: Math Question Answering
Systems were evaluated based on the percentage of correctly answered questions.
MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms
We introduce a new representation language to model precise operation programs corresponding to each math problem that aim to improve both the performance and the interpretability of the learned models.
A General Framework for Information Extraction using Dynamic Span Graphs
We introduce a general framework for several information extraction tasks that share span representations using dynamically constructed span graphs.
Ranked #1 on
Relation Extraction
on ACE 2004
(Cross Sentence metric)
Joint Entity and Relation Extraction
Named Entity Recognition (NER)
Text Generation from Knowledge Graphs with Graph Transformers
Generating texts which express complex ideas spanning multiple sentences requires a structured representation of their content (document plan), but these representations are prohibitively expensive to manually produce.
Ranked #6 on
KG-to-Text Generation
on AGENDA
ESPNetv2: A Light-weight, Power Efficient, and General Purpose Convolutional Neural Network
Compared to YOLOv2 on the MS-COCO object detection, ESPNetv2 delivers 4. 4% higher accuracy with 6x fewer FLOPs.
Ranked #41 on
Semantic Segmentation
on PASCAL VOC 2012 test
Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction
We introduce a multi-task setup of identifying and classifying entities, relations, and coreference clusters in scientific articles.
Ranked #7 on
Joint Entity and Relation Extraction
on SciERC
Coreference Resolution
Joint Entity and Relation Extraction
+1
Pyramidal Recurrent Unit for Language Modeling
We introduce the Pyramidal Recurrent Unit (PRU), which enables learning representations in high dimensional space with more generalization power and fewer parameters.
Semi-Supervised Event Extraction with Paraphrase Clusters
Supervised event extraction systems are limited in their accuracy due to the lack of available training data.
Scientific Relation Extraction with Selectively Incorporated Concept Embeddings
This paper describes our submission for the SemEval 2018 Task 7 shared task on semantic relation extraction and classification in scientific papers.
The UWNLP system at SemEval-2018 Task 7: Neural Relation Extraction Model with Selectively Incorporated Concept Embeddings
This paper describes our submission for SemEval 2018 Task 7 shared task on semantic relation extraction and classification in scientific papers.
Data-Driven Methods for Solving Algebra Word Problems
We explore contemporary, data-driven techniques for solving math word problems over recent large-scale datasets.
Phrase-Indexed Question Answering: A New Challenge for Scalable Document Comprehension
We formalize a new modular variant of current question answering tasks by enforcing complete independence of the document encoder from the question encoder.
ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation
We introduce a fast and efficient convolutional neural network, ESPNet, for semantic segmentation of high resolution images under resource constraints.
Ranked #48 on
Semantic Segmentation
on PASCAL VOC 2012 test
Identifying Most Walkable Direction for Navigation in an Outdoor Environment
We present an approach for identifying the most walkable direction for navigation using a hand-held camera.
Neural Speed Reading via Skim-RNN
Inspired by the principles of speed reading, we introduce Skim-RNN, a recurrent neural network (RNN) that dynamically decides to update only a small fraction of the hidden state for relatively unimportant input tokens.
Scientific Information Extraction with Semi-supervised Neural Tagging
This paper addresses the problem of extracting keyphrases from scientific articles and categorizing them as corresponding to a task, process, or material.
Are You Smarter Than a Sixth Grader? Textbook Question Answering for Multimodal Machine Comprehension
Our analysis shows that a significant portion of questions require complex parsing of the text and the diagrams and reasoning, indicating that our dataset is more complex compared to previous machine comprehension and visual question answering datasets.
Question Answering through Transfer Learning from Large Fine-grained Supervision Data
We show that the task of question answering (QA) can significantly benefit from the transfer learning of models trained on a different large, fine-grained QA dataset.
Bidirectional Attention Flow for Machine Comprehension
Machine comprehension (MC), answering a query about a given context paragraph, requires modeling complex interactions between the context and the query.
Ranked #4 on
Question Answering
on CNN / Daily Mail
A Theme-Rewriting Approach for Generating Algebra Word Problems
Texts present coherent stories that have a particular theme or overall setting, for example science fiction or western.
Query-Reduction Networks for Question Answering
In this paper, we study the problem of question answering when reasoning over multiple facts is required.
Ranked #2 on
Question Answering
on bAbi
A Task-Oriented Approach for Cost-Sensitive Recognition
With the recent progress in visual recognition, we have already started to see a surge of vision related real-world applications.
Disfluency Detection using a Bidirectional LSTM
We introduce a new approach for disfluency detection using a Bidirectional Long-Short Term Memory neural network (BLSTM).
A Diagram Is Worth A Dozen Images
We define syntactic parsing of diagrams as learning to infer DPGs for diagrams and study semantic interpretation and reasoning of diagrams in the context of diagram question answering.
Are Elephants Bigger than Butterflies? Reasoning about Sizes of Objects
In this paper, we introduce a method to automatically infer object sizes, leveraging visual and textual information from web.
Talking to the crowd: What do people react to in online discussions?
This paper addresses the question of how language use affects community reaction to comments in online discussion forums, and the relative importance of the message vs. the messenger.
Discriminative and Consistent Similarities in Instance-Level Multiple Instance Learning
In this paper we present a bottom-up method to instance-level Multiple Instance Learning (MIL) that learns to discover positive instances with globally constrained reasoning about local pairwise similarities.
Parsing Algebraic Word Problems into Equations
This paper formalizes the problem of solving multi-sentence algebraic word problems as that of generating and scoring equation trees.
