Search Results for author: Wen-tau Yih

Found 75 papers, 27 papers with code

Structured Prompt Tuning

no code implementations24 May 2022 Chi-Liang Liu, Hung-Yi Lee, Wen-tau Yih

We propose structured prompt tuning, a simple and effective method to improve prompt tuning.

On Continual Model Refinement in Out-of-Distribution Data Streams

no code implementations ACL 2022 Bill Yuchen Lin, Sida Wang, Xi Victoria Lin, Robin Jia, Lin Xiao, Xiang Ren, Wen-tau Yih

Real-world natural language processing (NLP) models need to be continually updated to fix the prediction errors in out-of-distribution (OOD) data streams while overcoming catastrophic forgetting.

Continual Learning

Autoregressive Search Engines: Generating Substrings as Document Identifiers

1 code implementation22 Apr 2022 Michele Bevilacqua, Giuseppe Ottaviano, Patrick Lewis, Wen-tau Yih, Sebastian Riedel, Fabio Petroni

Knowledge-intensive language tasks require NLP systems to both provide the correct answer and retrieve supporting evidence for it in a given corpus.

Information Retrieval

InCoder: A Generative Model for Code Infilling and Synthesis

1 code implementation12 Apr 2022 Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Wen-tau Yih, Luke Zettlemoyer, Mike Lewis

Our model is the first generative model that is able to directly perform zero-shot code infilling, which we evaluate on challenging tasks such as type inference, comment generation, and variable re-naming.

Program Synthesis

The Web Is Your Oyster -- Knowledge-Intensive NLP against a Very Large Web Corpus

2 code implementations18 Dec 2021 Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Dmytro Okhonko, Samuel Broscheit, Gautier Izacard, Patrick Lewis, Barlas Oğuz, Edouard Grave, Wen-tau Yih, Sebastian Riedel

In order to address increasing demands of real-world applications, the research for knowledge-intensive NLP (KI-NLP) should advance by capturing the challenges of a truly open-domain environment: web-scale knowledge, lack of structure, inconsistent quality and noise.

Common Sense Reasoning

Boosted Dense Retriever

no code implementations14 Dec 2021 Patrick Lewis, Barlas Oğuz, Wenhan Xiong, Fabio Petroni, Wen-tau Yih, Sebastian Riedel

DrBoost is trained in stages: each component model is learned sequentially and specialized by focusing only on retrieval mistakes made by the current ensemble.

Quantization

Simple Local Attentions Remain Competitive for Long-Context Tasks

1 code implementation14 Dec 2021 Wenhan Xiong, Barlas Oğuz, Anchit Gupta, Xilun Chen, Diana Liskovich, Omer Levy, Wen-tau Yih, Yashar Mehdad

Many NLP tasks require processing long contexts beyond the length limit of pretrained models.

UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning

1 code implementation ACL 2022 Yuning Mao, Lambert Mathias, Rui Hou, Amjad Almahairi, Hao Ma, Jiawei Han, Wen-tau Yih, Madian Khabsa

Recent parameter-efficient language model tuning (PELT) methods manage to match the performance of fine-tuning with much fewer trainable parameters and perform especially well when training data is limited.

Language Modelling Model Selection

Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?

1 code implementation13 Oct 2021 Xilun Chen, Kushal Lakhotia, Barlas Oğuz, Anchit Gupta, Patrick Lewis, Stan Peshterliev, Yashar Mehdad, Sonal Gupta, Wen-tau Yih

Despite their recent popularity and well-known advantages, dense retrievers still lag behind sparse methods such as BM25 in their ability to reliably match salient phrases and rare entities in the query and to generalize to out-of-domain data.

Open-Domain Question Answering Passage Retrieval

Domain-matched Pre-training Tasks for Dense Retrieval

1 code implementation28 Jul 2021 Barlas Oğuz, Kushal Lakhotia, Anchit Gupta, Patrick Lewis, Vladimir Karpukhin, Aleksandra Piktus, Xilun Chen, Sebastian Riedel, Wen-tau Yih, Sonal Gupta, Yashar Mehdad

Pre-training on larger datasets with ever increasing model size is now a proven recipe for increased performance across almost all NLP tasks.

 Ranked #1 on Passage Retrieval on Natural Questions (using extra training data)

Passage Retrieval

On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study

1 code implementation ACL 2021 Divyansh Kaushik, Douwe Kiela, Zachary C. Lipton, Wen-tau Yih

In adversarial data collection (ADC), a human workforce interacts with a model in real time, attempting to produce examples that elicit incorrect predictions.

Question Answering

RECONSIDER: Improved Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering

no code implementations NAACL 2021 Srinivasan Iyer, Sewon Min, Yashar Mehdad, Wen-tau Yih

State-of-the-art Machine Reading Comprehension (MRC) models for Open-domain Question Answering (QA) are typically trained for span selection using distantly supervised positive examples and heuristically retrieved negative examples.

Machine Reading Comprehension Open-Domain Question Answering +1

On Unifying Misinformation Detection

1 code implementation NAACL 2021 Nayeon Lee, Belinda Z. Li, Sinong Wang, Pascale Fung, Hao Ma, Wen-tau Yih, Madian Khabsa

In this paper, we introduce UnifiedM2, a general-purpose misinformation model that jointly models multiple domains of misinformation with a single, unified setup.

Few-Shot Learning Misinformation

FiD-Ex: Improving Sequence-to-Sequence Models for Extractive Rationale Generation

no code implementations EMNLP 2021 Kushal Lakhotia, Bhargavi Paranjape, Asish Ghoshal, Wen-tau Yih, Yashar Mehdad, Srinivasan Iyer

Natural language (NL) explanations of model predictions are gaining popularity as a means to understand and verify decisions made by large black-box pre-trained models, for NLP tasks such as Question Answering (QA) and Fact Verification.

Fact Verification Question Answering

RECONSIDER: Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering

1 code implementation21 Oct 2020 Srinivasan Iyer, Sewon Min, Yashar Mehdad, Wen-tau Yih

State-of-the-art Machine Reading Comprehension (MRC) models for Open-domain Question Answering (QA) are typically trained for span selection using distantly supervised positive examples and heuristically retrieved negative examples.

Machine Reading Comprehension Open-Domain Question Answering +1

Efficient One-Pass End-to-End Entity Linking for Questions

3 code implementations EMNLP 2020 Belinda Z. Li, Sewon Min, Srinivasan Iyer, Yashar Mehdad, Wen-tau Yih

We present ELQ, a fast end-to-end entity linking model for questions, which uses a biencoder to jointly perform mention detection and linking in one pass.

Entity Linking Question Answering

Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval

1 code implementation ICLR 2021 Wenhan Xiong, Xiang Lorraine Li, Srini Iyer, Jingfei Du, Patrick Lewis, William Yang Wang, Yashar Mehdad, Wen-tau Yih, Sebastian Riedel, Douwe Kiela, Barlas Oğuz

We propose a simple and efficient multi-hop dense retrieval approach for answering complex open-domain questions, which achieves state-of-the-art performance on two multi-hop datasets, HotpotQA and multi-evidence FEVER.

Question Answering

Open-Domain Question Answering

no code implementations ACL 2020 Danqi Chen, Wen-tau Yih

This tutorial provides a comprehensive and coherent overview of cutting-edge research in open-domain question answering (QA), the task of answering questions using a large collection of documents of diversified topics.

Open-Domain Question Answering

Language Models as Fact Checkers?

no code implementations WS 2020 Nayeon Lee, Belinda Z. Li, Sinong Wang, Wen-tau Yih, Hao Ma, Madian Khabsa

Recent work has suggested that language models (LMs) store both common-sense and factual knowledge learned from pre-training data.

Common Sense Reasoning Language Modelling +1

An Imitation Game for Learning Semantic Parsers from User Interaction

1 code implementation EMNLP 2020 Ziyu Yao, Yiqi Tang, Wen-tau Yih, Huan Sun, Yu Su

Despite the widely successful applications, bootstrapping and fine-tuning semantic parsers are still a tedious process with challenges such as costly data annotation and privacy risks.

Imitation Learning Text-To-Sql

Dense Passage Retrieval for Open-Domain Question Answering

12 code implementations EMNLP 2020 Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, Wen-tau Yih

Open-domain question answering relies on efficient passage retrieval to select candidate contexts, where traditional sparse vector space models, such as TF-IDF or BM25, are the de facto method.

Open-Domain Question Answering Passage Retrieval

Unsupervised Question Decomposition for Question Answering

2 code implementations EMNLP 2020 Ethan Perez, Patrick Lewis, Wen-tau Yih, Kyunghyun Cho, Douwe Kiela

We aim to improve question answering (QA) by decomposing hard questions into simpler sub-questions that existing QA systems are capable of answering.

Question Answering

Model-based Interactive Semantic Parsing: A Unified Framework and A Text-to-SQL Case Study

2 code implementations IJCNLP 2019 Ziyu Yao, Yu Su, Huan Sun, Wen-tau Yih

As a promising paradigm, interactive semantic parsing has shown to improve both semantic parsing accuracy and user confidence in the results.

Semantic Parsing Text-To-Sql

Everything Happens for a Reason: Discovering the Purpose of Actions in Procedural Text

no code implementations IJCNLP 2019 Bhavana Dalvi Mishra, Niket Tandon, Antoine Bosselut, Wen-tau Yih, Peter Clark

Our goal is to better comprehend procedural text, e. g., a paragraph about photosynthesis, by not only predicting what happens, but why some actions need to happen before others.

Reading Comprehension

Be Consistent! Improving Procedural Text Comprehension using Label Consistency

1 code implementation NAACL 2019 Xinya Du, Bhavana Dalvi Mishra, Niket Tandon, Antoine Bosselut, Wen-tau Yih, Peter Clark, Claire Cardie

Our goal is procedural text comprehension, namely tracking how the properties of entities (e. g., their location) change with time given a procedural text (e. g., a paragraph about photosynthesis, a recipe).

Reading Comprehension

QuaRel: A Dataset and Models for Answering Questions about Qualitative Relationships

no code implementations20 Nov 2018 Oyvind Tafjord, Peter Clark, Matt Gardner, Wen-tau Yih, Ashish Sabharwal

Many natural language questions require recognizing and reasoning with qualitative relationships (e. g., in science, economics, and medicine), but are challenging to answer with corpus-based methods.

Semantic Parsing

FlowQA: Grasping Flow in History for Conversational Machine Comprehension

1 code implementation ICLR 2019 Hsin-Yuan Huang, Eunsol Choi, Wen-tau Yih

Conversational machine comprehension requires the understanding of the conversation history, such as previous question/answer pairs, the document context, and the current question.

Question Answering Reading Comprehension

QuAC: Question Answering in Context

no code implementations EMNLP 2018 Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, Luke Zettlemoyer

We present QuAC, a dataset for Question Answering in Context that contains 14K information-seeking QA dialogs (100K questions in total).

Question Answering Reading Comprehension

Policy Shaping and Generalized Update Equations for Semantic Parsing from Denotations

no code implementations EMNLP 2018 Dipendra Misra, Ming-Wei Chang, Xiaodong He, Wen-tau Yih

Semantic parsing from denotations faces two key challenges in model training: (1) given only the denotations (e. g., answers), search for good candidate semantic parses, and (2) choose the best model update algorithm.

Question Answering Semantic Parsing

Reasoning about Actions and State Changes by Injecting Commonsense Knowledge

1 code implementation EMNLP 2018 Niket Tandon, Bhavana Dalvi Mishra, Joel Grus, Wen-tau Yih, Antoine Bosselut, Peter Clark

Comprehending procedural text, e. g., a paragraph describing photosynthesis, requires modeling actions and the state changes they produce, so that questions about entities at different timepoints can be answered.

Reading Comprehension Structured Prediction

Dissecting Contextual Word Embeddings: Architecture and Representation

no code implementations EMNLP 2018 Matthew E. Peters, Mark Neumann, Luke Zettlemoyer, Wen-tau Yih

Contextual word representations derived from pre-trained bidirectional language models (biLMs) have recently been shown to provide significant improvements to the state of the art for a wide range of NLP tasks.

Word Embeddings

QuAC : Question Answering in Context

no code implementations21 Aug 2018 Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, Luke Zettlemoyer

We present QuAC, a dataset for Question Answering in Context that contains 14K information-seeking QA dialogs (100K questions in total).

Question Answering Reading Comprehension

Tracking State Changes in Procedural Text: A Challenge Dataset and Models for Process Paragraph Comprehension

no code implementations NAACL 2018 Bhavana Dalvi Mishra, Lifu Huang, Niket Tandon, Wen-tau Yih, Peter Clark

The new dataset, ProPara, is the first to contain natural (rather than machine-generated) text about a changing world along with a full annotation of entity states (location and existence) during those changes (81k datapoints).

Procedural Text Understanding

Maximum Margin Reward Networks for Learning from Explicit and Implicit Supervision

no code implementations EMNLP 2017 Haoruo Peng, Ming-Wei Chang, Wen-tau Yih

Neural networks have achieved state-of-the-art performance on several structured-output prediction tasks, trained in a fully supervised fashion.

Dependency Parsing Named Entity Recognition +2

Search-based Neural Structured Learning for Sequential Question Answering

no code implementations ACL 2017 Mohit Iyyer, Wen-tau Yih, Ming-Wei Chang

Recent work in semantic parsing for question answering has focused on long and complicated questions, many of which would seem unnatural if asked in a normal conversation between two humans.

Question Answering Semantic Parsing

NLP for Precision Medicine

no code implementations ACL 2017 Hoifung Poon, Chris Quirk, Kristina Toutanova, Wen-tau Yih

We will introduce precision medicine and showcase the vast opportunities for NLP in this burgeoning field with great societal impact.

Decision Making Entity Linking +2

A Knowledge-Grounded Neural Conversation Model

2 code implementations7 Feb 2017 Marjan Ghazvininejad, Chris Brockett, Ming-Wei Chang, Bill Dolan, Jianfeng Gao, Wen-tau Yih, Michel Galley

We generalize the widely-used Seq2Seq approach by conditioning responses on both conversation history and external "facts", allowing the model to be versatile and applicable in an open-domain setting.

Slot Filling

Answering Complicated Question Intents Expressed in Decomposed Question Sequences

no code implementations4 Nov 2016 Mohit Iyyer, Wen-tau Yih, Ming-Wei Chang

Recent work in semantic parsing for question answering has focused on long and complicated questions, many of which would seem unnatural if asked in a normal conversation between two humans.

Question Answering Semantic Parsing

Basic Reasoning with Tensor Product Representations

no code implementations12 Jan 2016 Paul Smolensky, Moontae Lee, Xiaodong He, Wen-tau Yih, Jianfeng Gao, Li Deng

In this paper we present the initial development of a general theory for mapping inference in predicate logic to computation over Tensor Product Representations (TPRs; Smolensky (1990), Smolensky & Legendre (2006)).

Question Answering

Embedding Entities and Relations for Learning and Inference in Knowledge Bases

8 code implementations20 Dec 2014 Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, Li Deng

We consider learning representations of entities and relations in KBs using the neural-embedding approach.

Link Prediction

Learning Multi-Relational Semantics Using Neural-Embedding Models

no code implementations14 Nov 2014 Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, Li Deng

In this paper we present a unified framework for modeling multi-relational representations, scoring, and learning, and conduct an empirical study of several recent multi-relational embedding models under the framework.

Knowledge Base Completion

Learning Semantic Representations for the Phrase Translation Model

no code implementations28 Nov 2013 Jianfeng Gao, Xiaodong He, Wen-tau Yih, Li Deng

The results show that the new semantic-based phrase translation model significantly improves the performance of a state-of-the-art phrase-based statistical machine translation sys-tem, leading to a gain of 0. 7-1. 0 BLEU points.

Learning Semantic Representations Machine Translation +1

Dual Coordinate Descent Algorithms for Efficient Large Margin Structured Prediction

no code implementations TACL 2013 Ming-Wei Chang, Wen-tau Yih

Due to the nature of complex NLP problems, structured prediction algorithms have been important modeling tools for a wide range of tasks.

Dependency Parsing Document Summarization +6

Cannot find the paper you are looking for? You can Submit a new open access paper.