Search Results for author:
Found 95 papers, 42 papers with code
Dual Coordinate Descent Algorithms for Efficient Large Margin Structured Prediction
Due to the nature of complex NLP problems, structured prediction algorithms have been important modeling tools for a wide range of tasks.
Learning Semantic Representations for the Phrase Translation Model
The results show that the new semantic-based phrase translation model significantly improves the performance of a state-of-the-art phrase-based statistical machine translation sys-tem, leading to a gain of 0. 7-1. 0 BLEU points.
Learning Multi-Relational Semantics Using Neural-Embedding Models
In this paper we present a unified framework for modeling multi-relational representations, scoring, and learning, and conduct an empirical study of several recent multi-relational embedding models under the framework.
Embedding Entities and Relations for Learning and Inference in Knowledge Bases
We consider learning representations of entities and relations in KBs using the neural-embedding approach.
Ranked #10 on
Link Prediction
on UMLS
Reasoning in Vector Space: An Exploratory Study of Question Answering
Question answering tasks have shown remarkable progress with distributed vector representation.
Basic Reasoning with Tensor Product Representations
In this paper we present the initial development of a general theory for mapping inference in predicate logic to computation over Tensor Product Representations (TPRs; Smolensky (1990), Smolensky & Legendre (2006)).
Answering Complicated Question Intents Expressed in Decomposed Question Sequences
Recent work in semantic parsing for question answering has focused on long and complicated questions, many of which would seem unnatural if asked in a normal conversation between two humans.
A Knowledge-Grounded Neural Conversation Model
We generalize the widely-used Seq2Seq approach by conditioning responses on both conversation history and external "facts", allowing the model to be versatile and applicable in an open-domain setting.
Search-based Neural Structured Learning for Sequential Question Answering
Recent work in semantic parsing for question answering has focused on long and complicated questions, many of which would seem unnatural if asked in a normal conversation between two humans.
NLP for Precision Medicine
We will introduce precision medicine and showcase the vast opportunities for NLP in this burgeoning field with great societal impact.
Cross-Sentence N-ary Relation Extraction with Graph LSTMs
Past work in relation extraction has focused on binary relations in single sentences.
Maximum Margin Reward Networks for Learning from Explicit and Implicit Supervision
Neural networks have achieved state-of-the-art performance on several structured-output prediction tasks, trained in a fully supervised fashion.
Natural Language to Structured Query Generation via Meta-Learning
In conventional supervised training, a model is trained to fit all the training examples.
Ranked #7 on
Code Generation
on WikiSQL
Tracking State Changes in Procedural Text: A Challenge Dataset and Models for Process Paragraph Comprehension
The new dataset, ProPara, is the first to contain natural (rather than machine-generated) text about a changing world along with a full annotation of entity states (location and existence) during those changes (81k datapoints).
Ranked #4 on
Procedural Text Understanding
on ProPara
QuAC : Question Answering in Context
We present QuAC, a dataset for Question Answering in Context that contains 14K information-seeking QA dialogs (100K questions in total).
Dissecting Contextual Word Embeddings: Architecture and Representation
Contextual word representations derived from pre-trained bidirectional language models (biLMs) have recently been shown to provide significant improvements to the state of the art for a wide range of NLP tasks.
Reasoning about Actions and State Changes by Injecting Commonsense Knowledge
Comprehending procedural text, e. g., a paragraph describing photosynthesis, requires modeling actions and the state changes they produce, so that questions about entities at different timepoints can be answered.
Policy Shaping and Generalized Update Equations for Semantic Parsing from Denotations
Semantic parsing from denotations faces two key challenges in model training: (1) given only the denotations (e. g., answers), search for good candidate semantic parses, and (2) choose the best model update algorithm.
QuAC: Question Answering in Context
We present QuAC, a dataset for Question Answering in Context that contains 14K information-seeking QA dialogs (100K questions in total).
FlowQA: Grasping Flow in History for Conversational Machine Comprehension
Conversational machine comprehension requires the understanding of the conversation history, such as previous question/answer pairs, the document context, and the current question.
Ranked #1 on
Question Answering
on QuAC
QuaRel: A Dataset and Models for Answering Questions about Qualitative Relationships
Many natural language questions require recognizing and reasoning with qualitative relationships (e. g., in science, economics, and medicine), but are challenging to answer with corpus-based methods.
Be Consistent! Improving Procedural Text Comprehension using Label Consistency
Our goal is procedural text comprehension, namely tracking how the properties of entities (e. g., their location) change with time given a procedural text (e. g., a paragraph about photosynthesis, a recipe).
Everything Happens for a Reason: Discovering the Purpose of Actions in Procedural Text
Our goal is to better comprehend procedural text, e. g., a paragraph about photosynthesis, by not only predicting what happens, but why some actions need to happen before others.
Model-based Interactive Semantic Parsing: A Unified Framework and A Text-to-SQL Case Study
As a promising paradigm, interactive semantic parsing has shown to improve both semantic parsing accuracy and user confidence in the results.
Unsupervised Question Decomposition for Question Answering
We aim to improve question answering (QA) by decomposing hard questions into simpler sub-questions that existing QA systems are capable of answering.
Dense Passage Retrieval for Open-Domain Question Answering
Open-domain question answering relies on efficient passage retrieval to select candidate contexts, where traditional sparse vector space models, such as TF-IDF or BM25, are the de facto method.
Ranked #1 on
Question Answering
on NaturalQA
An Imitation Game for Learning Semantic Parsers from User Interaction
Despite the widely successful applications, bootstrapping and fine-tuning semantic parsers are still a tedious process with challenges such as costly data annotation and privacy risks.
TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data
Recent years have witnessed the burgeoning of pretrained language models (LMs) for text-based natural language (NL) understanding tasks.
Ranked #10 on
Text-To-SQL
on spider
(Exact Match Accuracy (Dev) metric)
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Large pre-trained language models have been shown to store factual knowledge in their parameters, and achieve state-of-the-art results when fine-tuned on downstream NLP tasks.
Ranked #4 on
Question Answering
on WebQuestions
Language Models as Fact Checkers?
Recent work has suggested that language models (LMs) store both common-sense and factual knowledge learned from pre-training data.
Open-Domain Question Answering
This tutorial provides a comprehensive and coherent overview of cutting-edge research in open-domain question answering (QA), the task of answering questions using a large collection of documents of diversified topics.
Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval
We propose a simple and efficient multi-hop dense retrieval approach for answering complex open-domain questions, which achieves state-of-the-art performance on two multi-hop datasets, HotpotQA and multi-evidence FEVER.
Ranked #14 on
Question Answering
on HotpotQA
Efficient One-Pass End-to-End Entity Linking for Questions
We present ELQ, a fast end-to-end entity linking model for questions, which uses a biencoder to jointly perform mention detection and linking in one pass.
RECONSIDER: Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering
State-of-the-art Machine Reading Comprehension (MRC) models for Open-domain Question Answering (QA) are typically trained for span selection using distantly supervised positive examples and heuristically retrieved negative examples.
Joint Verification and Reranking for Open Fact Checking Over Tables
Structured information is an important knowledge source for automatic verification of factual claims.
FiD-Ex: Improving Sequence-to-Sequence Models for Extractive Rationale Generation
Natural language (NL) explanations of model predictions are gaining popularity as a means to understand and verify decisions made by large black-box pre-trained models, for NLP tasks such as Question Answering (QA) and Fact Verification.
Studying Strategically: Learning to Mask for Closed-book QA
Thus, our policy packs task-relevant knowledge into the parameters of a language model.
NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
We review the EfficientQA competition from NeurIPS 2020.
Multi-task Retrieval for Knowledge-Intensive Tasks
Retrieving relevant contexts from a large corpus is a crucial step for tasks such as open-domain question answering and fact checking.
On Unifying Misinformation Detection
In this paper, we introduce UnifiedM2, a general-purpose misinformation model that jointly models multiple domains of misinformation with a single, unified setup.
On the Influence of Masking Policies in Intermediate Pre-training
Current NLP models are predominantly trained through a two-stage "pre-train then fine-tune" pipeline.
RECONSIDER: Improved Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering
State-of-the-art Machine Reading Comprehension (MRC) models for Open-domain Question Answering (QA) are typically trained for span selection using distantly supervised positive examples and heuristically retrieved negative examples.
On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study
In adversarial data collection (ADC), a human workforce interacts with a model in real time, attempting to produce examples that elicit incorrect predictions.
Domain-matched Pre-training Tasks for Dense Retrieval
Pre-training on larger datasets with ever increasing model size is now a proven recipe for increased performance across almost all NLP tasks.
Ranked #2 on
Passage Retrieval
on Natural Questions
(using extra training data)
Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?
Despite their recent popularity and well-known advantages, dense retrievers still lag behind sparse methods such as BM25 in their ability to reliably match salient phrases and rare entities in the query and to generalize to out-of-domain data.
Ranked #2 on
Passage Retrieval
on EntityQuestions
UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning
Recent parameter-efficient language model tuning (PELT) methods manage to match the performance of fine-tuning with much fewer trainable parameters and perform especially well when training data is limited.
CCQA: A New Web-Scale Question Answering Dataset for Model Pre-Training
Consequently, we propose a novel QA dataset based on the Common Crawl project in this paper.
Simple Local Attentions Remain Competitive for Long-Context Tasks
Many NLP tasks require processing long contexts beyond the length limit of pretrained models.
Boosted Dense Retriever
DrBoost is trained in stages: each component model is learned sequentially and specialized by focusing only on retrieval mistakes made by the current ensemble.
The Web Is Your Oyster -- Knowledge-Intensive NLP against a Very Large Web Corpus
In order to address increasing demands of real-world applications, the research for knowledge-intensive NLP (KI-NLP) should advance by capturing the challenges of a truly open-domain environment: web-scale knowledge, lack of structure, inconsistent quality and noise.
InCoder: A Generative Model for Code Infilling and Synthesis
Our model is the first generative model that is able to directly perform zero-shot code infilling, which we evaluate on challenging tasks such as type inference, comment generation, and variable re-naming.
Ranked #84 on
Code Generation
on MBPP
Improving Passage Retrieval with Zero-Shot Question Generation
We propose a simple and effective re-ranking method for improving passage retrieval in open question answering.
DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings
We propose DiffCSE, an unsupervised contrastive learning framework for learning sentence embeddings.
Ranked #13 on
Semantic Textual Similarity
on STS16
Autoregressive Search Engines: Generating Substrings as Document Identifiers
Knowledge-intensive language tasks require NLP systems to both provide the correct answer and retrieve supporting evidence for it in a given corpus.
On Continual Model Refinement in Out-of-Distribution Data Streams
Real-world natural language processing (NLP) models need to be continually updated to fix the prediction errors in out-of-distribution (OOD) data streams while overcoming catastrophic forgetting.
Structured Prompt Tuning
We propose structured prompt tuning, a simple and effective method to improve prompt tuning.
Adapting Pretrained Text-to-Text Models for Long Text Sequences
We present an empirical study of adapting an existing pretrained text-to-text model for long-sequence inputs.
Ranked #1 on
Text Summarization
on QMSum
RoMQA: A Benchmark for Robust, Multi-evidence, Multi-answer Question Answering
Moreover, existing models are not robust to variations in question constraints, but can be made more robust by tuning on clusters of related questions.
Task-aware Retrieval with Instructions
We study the problem of retrieval with instructions, where users of a retrieval system explicitly describe their intent along with their queries.
CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrieval
In this paper, we unify different multi-vector retrieval models from a token routing viewpoint and propose conditional token interaction via dynamic lexical routing, namely CITADEL, for efficient and effective multi-vector retrieval.
Retrieval-Augmented Multimodal Language Modeling
To integrate knowledge in a more scalable and modular way, we propose a retrieval-augmented multimodal model, which enables a base multimodal model (generator) to refer to relevant text and images fetched by a retriever from external memory (e. g., documents on the web).
Ranked #7 on
Image Captioning
on MS COCO
Coder Reviewer Reranking for Code Generation
Sampling diverse programs from a code language model and reranking with model likelihood is a popular method for code generation but it is prone to preferring degenerate solutions.
Ranked #22 on
Code Generation
on MBPP
Nonparametric Masked Language Modeling
Existing language models (LMs) predict tokens with a softmax over a finite vocabulary, which can make it difficult to predict rare tokens or phrases.
One Embedder, Any Task: Instruction-Finetuned Text Embeddings
Our analysis suggests that INSTRUCTOR is robust to changes in instructions, and that instruction finetuning mitigates the challenge of training a single model on diverse datasets.
REPLUG: Retrieval-Augmented Black-Box Language Models
We introduce REPLUG, a retrieval-augmented language modeling framework that treats the language model (LM) as a black box and augments it with a tuneable retrieval model.
Ranked #9 on
Question Answering
on Natural Questions
How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrieval
We hence propose a new DA approach with diverse queries and sources of supervision to progressively train a generalizable DR. As a result, DRAGON, our dense retriever trained with diverse augmentation, is the first BERT-base-sized DR to achieve state-of-the-art effectiveness in both supervised and zero-shot evaluations and even competes with models using more complex late interaction (ColBERTv2 and SPLADE++).
LEVER: Learning to Verify Language-to-Code Generation with Execution
The advent of large language models trained on code (code LLMs) has led to significant progress in language-to-code generation.
Ranked #2 on
Semantic Parsing
on spider
VideoOFA: Two-Stage Pre-Training for Video-to-Text Generation
We propose a new two-stage pre-training framework for video-to-text generation tasks such as video captioning and video question answering: A generative encoder-decoder model is first jointly pre-trained on massive image-text data to learn fundamental vision-language concepts, and then adapted to video data in an intermediate video-text pre-training stage to learn video-specific skills such as spatio-temporal reasoning.
Large Language Model Programs
In recent years, large pre-trained language models (LLMs) have demonstrated the ability to follow instructions and perform novel tasks from a few examples.
Learning to Simulate Natural Language Feedback for Interactive Semantic Parsing
In this work, we propose a new task of simulating NL feedback for interactive semantic parsing.
FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
Evaluating the factuality of long-form text generated by large language models (LMs) is non-trivial because (1) generations often contain a mixture of supported and unsupported pieces of information, making binary judgments of quality inadequate, and (2) human evaluation is time-consuming and costly.
Few-Shot Data Synthesis for Open Domain Multi-Hop Question Answering
Few-shot learning for open domain multi-hop question answering typically relies on the incontext learning capability of large language models (LLMs).
Expand, Rerank, and Retrieve: Query Reranking for Open-Domain Question Answering
We propose EAR, a query Expansion And Reranking approach for improving passage retrieval, with the application to open-domain question answering.
Instruction-tuned Language Models are Better Knowledge Learners
The standard recipe for doing so involves continued pre-training on new documents followed by instruction-tuning on question-answer (QA) pairs.
Reliable, Adaptable, and Attributable Language Models with Retrieval
Parametric language models (LMs), which are trained on vast amounts of web data, exhibit remarkable flexibility and capability.
Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM
We investigate efficient methods for training Large Language Models (LLMs) to possess capabilities in multiple specialized domains, such as coding, math reasoning and world knowledge.
Ranked #30 on
Question Answering
on TriviaQA
