Search Results for author: Kenton Lee

Found 37 papers, 21 papers with code

CapWAP: Image Captioning with a Purpose

no code implementations EMNLP 2020 Adam Fisch, Kenton Lee, Ming-Wei Chang, Jonathan Clark, Regina Barzilay

In this task, we use question-answer (QA) pairs{---}a natural expression of information need{---}from users, instead of reference captions, for both training and post-inference evaluation.

Image Captioning Question Answering +1

Retrieval Augmented Language Model Pre-Training

no code implementations ICML 2020 Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, Ming-Wei Chang

Language model pre-training has been shown to capture a surprising amount of world knowledge, crucial for NLP tasks such as question answering.

Language Modelling Open-Domain Question Answering

Revisiting the Primacy of English in Zero-shot Cross-lingual Transfer

no code implementations30 Jun 2021 Iulia Turc, Kenton Lee, Jacob Eisenstein, Ming-Wei Chang, Kristina Toutanova

Zero-shot cross-lingual transfer is emerging as a practical solution: pre-trained models later fine-tuned on one transfer language exhibit surprising performance when tested on many target languages.

Question Answering Zero-Shot Cross-Lingual Transfer

Joint Passage Ranking for Diverse Multi-Answer Retrieval

no code implementations EMNLP 2021 Sewon Min, Kenton Lee, Ming-Wei Chang, Kristina Toutanova, Hannaneh Hajishirzi

We study multi-answer retrieval, an under-explored problem that requires retrieving passages to cover multiple distinct answers for a given question.

Passage Retrieval Question Answering

Neural Data Augmentation via Example Extrapolation

1 code implementation2 Feb 2021 Kenton Lee, Kelvin Guu, Luheng He, Tim Dozat, Hyung Won Chung

In many applications of machine learning, certain categories of examples may be underrepresented in the training data, causing systems to underperform on such "few-shot" cases at test time.

Data Augmentation Few-Shot Learning +3

CapWAP: Captioning with a Purpose

1 code implementation9 Nov 2020 Adam Fisch, Kenton Lee, Ming-Wei Chang, Jonathan H. Clark, Regina Barzilay

In this task, we use question-answer (QA) pairs---a natural expression of information need---from users, instead of reference captions, for both training and post-inference evaluation.

Image Captioning Question Answering +1

Exploring Unexplored Generalization Challenges for Cross-Database Semantic Parsing

no code implementations ACL 2020 Alane Suhr, Ming-Wei Chang, Peter Shaw, Kenton Lee

We study the task of cross-database semantic parsing (XSP), where a system that maps natural language utterances to executable SQL queries is evaluated on databases unseen during training.

Semantic Parsing

Probabilistic Assumptions Matter: Improved Models for Distantly-Supervised Document-Level Question Answering

1 code implementation ACL 2020 Hao Cheng, Ming-Wei Chang, Kenton Lee, Kristina Toutanova

We address the problem of extractive question answering using document-level distant super-vision, pairing questions and relevant documents with answer strings.

Question Answering

Contextualized Representations Using Textual Encyclopedic Knowledge

no code implementations24 Apr 2020 Mandar Joshi, Kenton Lee, Yi Luan, Kristina Toutanova

We present a method to represent input texts by contextualizing them jointly with dynamically retrieved textual encyclopedic background knowledge from multiple documents.

Language Modelling Reading Comprehension

REALM: Retrieval-Augmented Language Model Pre-Training

4 code implementations10 Feb 2020 Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, Ming-Wei Chang

Language model pre-training has been shown to capture a surprising amount of world knowledge, crucial for NLP tasks such as question answering.

Language Modelling Open-Domain Question Answering

Giving BERT a Calculator: Finding Operations and Arguments with Reading Comprehension

no code implementations IJCNLP 2019 Daniel Andor, Luheng He, Kenton Lee, Emily Pitler

Reading comprehension models have been successfully applied to extractive text answers, but it is unclear how best to generalize these models to abstractive numerical answers.

Question Answering Reading Comprehension

Well-Read Students Learn Better: On the Importance of Pre-training Compact Models

42 code implementations ICLR 2020 Iulia Turc, Ming-Wei Chang, Kenton Lee, Kristina Toutanova

Recent developments in natural language representations have been accompanied by large and expensive models that leverage vast amounts of general-domain text through self-supervised pre-training.

Knowledge Distillation Language Modelling +2

Zero-Shot Entity Linking by Reading Entity Descriptions

3 code implementations ACL 2019 Lajanugen Logeswaran, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, Jacob Devlin, Honglak Lee

First, we show that strong reading comprehension models pre-trained on large unlabeled data can be used to generalize to unseen entities.

Entity Linking Reading Comprehension

Latent Retrieval for Weakly Supervised Open Domain Question Answering

2 code implementations ACL 2019 Kenton Lee, Ming-Wei Chang, Kristina Toutanova

We show for the first time that it is possible to jointly learn the retriever and reader from question-answer string pairs and without any IR system.

Information Retrieval Open-Domain Question Answering

BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions

1 code implementation NAACL 2019 Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, Kristina Toutanova

In this paper we study yes/no questions that are naturally occurring --- meaning that they are generated in unprompted and unconstrained settings.

Reading Comprehension Transfer Learning

Language Model Pre-training for Hierarchical Document Representations

no code implementations ICLR 2019 Ming-Wei Chang, Kristina Toutanova, Kenton Lee, Jacob Devlin

Hierarchical neural architectures are often used to capture long-distance dependencies and have been applied to many document-level tasks such as summarization, document segmentation, and sentiment analysis.

Document Summarization Extractive Document Summarization +4

A BERT Baseline for the Natural Questions

3 code implementations24 Jan 2019 Chris Alberti, Kenton Lee, Michael Collins

This technical note describes a new baseline for the Natural Questions.

Ranked #2 on Question Answering on Natural Questions (F1 (Long) metric)

Question Answering

Improving Span-based Question Answering Systems with Coarsely Labeled Data

no code implementations5 Nov 2018 Hao Cheng, Ming-Wei Chang, Kenton Lee, Ankur Parikh, Michael Collins, Kristina Toutanova

We study approaches to improve fine-grained short answer Question Answering models by integrating coarse-grained data annotated for paragraph-level relevance and show that coarsely annotated data can bring significant performance gains.

Multi-Task Learning Question Answering

Syntactic Scaffolds for Semantic Structures

1 code implementation EMNLP 2018 Swabha Swayamdipta, Sam Thomson, Kenton Lee, Luke Zettlemoyer, Chris Dyer, Noah A. Smith

We introduce the syntactic scaffold, an approach to incorporating syntactic information into semantic tasks.

Coreference Resolution

Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling

1 code implementation ACL 2018 Luheng He, Kenton Lee, Omer Levy, Luke Zettlemoyer

Recent BIO-tagging-based neural semantic role labeling models are very high performing, but assume gold predicates as part of the input and cannot incorporate span-level features.

Semantic Role Labeling

Long Short-Term Memory as a Dynamically Computed Element-wise Weighted Sum

no code implementations ACL 2018 Omer Levy, Kenton Lee, Nicholas FitzGerald, Luke Zettlemoyer

LSTMs were introduced to combat vanishing gradients in simple RNNs by augmenting them with gated additive recurrent connections.

Deep contextualized word representations

44 code implementations NAACL 2018 Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, Luke Zettlemoyer

We introduce a new type of deep contextualized word representation that models both (1) complex characteristics of word use (e. g., syntax and semantics), and (2) how these uses vary across linguistic contexts (i. e., to model polysemy).

Ranked #2 on Citation Intent Classification on ACL-ARC (using extra training data)

Citation Intent Classification Conversational Response Selection +7

End-to-end Neural Coreference Resolution

4 code implementations EMNLP 2017 Kenton Lee, Luheng He, Mike Lewis, Luke Zettlemoyer

We introduce the first end-to-end coreference resolution model and show that it significantly outperforms all previous work without using a syntactic parser or hand-engineered mention detector.

Coreference Resolution

Deep Semantic Role Labeling: What Works and What's Next

1 code implementation ACL 2017 Luheng He, Kenton Lee, Mike Lewis, Luke Zettlemoyer

We introduce a new deep learning model for semantic role labeling (SRL) that significantly improves the state of the art, along with detailed analyses to reveal its strengths and limitations.

Predicate Detection

Recurrent Additive Networks

2 code implementations21 May 2017 Kenton Lee, Omer Levy, Luke Zettlemoyer

We introduce recurrent additive networks (RANs), a new gated RNN which is distinguished by the use of purely additive latent state updates.

Language Modelling

Learning Recurrent Span Representations for Extractive Question Answering

2 code implementations4 Nov 2016 Kenton Lee, Shimi Salant, Tom Kwiatkowski, Ankur Parikh, Dipanjan Das, Jonathan Berant

In this paper, we focus on this answer extraction task, presenting a novel model architecture that efficiently builds fixed length representations of all spans in the evidence document with a recurrent network.

Answer Selection Natural Language Understanding +1

Global Neural CCG Parsing with Optimality Guarantees

1 code implementation EMNLP 2016 Kenton Lee, Mike Lewis, Luke Zettlemoyer

We introduce the first global recursive neural parsing model with optimality guarantees during decoding.

Cannot find the paper you are looking for? You can Submit a new open access paper.