Search Results for author: Mohit Iyyer

Found 59 papers, 36 papers with code

IGA: An Intent-Guided Authoring Assistant

no code implementations EMNLP 2021 Simeng Sun, Wenlong Zhao, Varun Manjunatha, Rajiv Jain, Vlad Morariu, Franck Dernoncourt, Balaji Vasan Srinivasan, Mohit Iyyer

While large-scale pretrained language models have significantly improved writing assistance functionalities such as autocomplete, more complex and controllable writing assistants have yet to be explored.

Language Modelling Pretrained Language Models

Unsupervised Parsing with S-DIORA: Single Tree Encoding for Deep Inside-Outside Recursive Autoencoders

no code implementations EMNLP 2020 Andrew Drozdov, Subendhu Rongali, Yi-Pei Chen, Tim O{'}Gorman, Mohit Iyyer, Andrew McCallum

The deep inside-outside recursive autoencoder (DIORA; Drozdov et al. 2019) is a self-supervised neural model that learns to induce syntactic tree structures for input sentences *without access to labeled training data*.

Constituency Parsing

How Much Do Modifications to Transformer Language Models Affect Their Ability to Learn Linguistic Knowledge?

no code implementations insights (ACL) 2022 Simeng Sun, Brian Dillon, Mohit Iyyer

Recent progress in large pretrained language models (LMs) has led to a growth of analyses examining what kinds of linguistic knowledge are encoded by these models.

Pretrained Language Models

Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generation

no code implementations25 May 2022 Tu Vu, Aditya Barua, Brian Lester, Daniel Cer, Mohit Iyyer, Noah Constant

In this paper, we explore the challenging problem of performing a generative task (i. e., summarization) in a target language when labeled data is only available in English.

Cross-Lingual Transfer Machine Translation +1

Modeling Exemplification in Long-form Question Answering via Retrieval

no code implementations19 May 2022 Shufan Wang, Fangyuan Xu, Laure Thompson, Eunsol Choi, Mohit Iyyer

We show that not only do state-of-the-art LFQA models struggle to generate relevant examples, but also that standard evaluation metrics such as ROUGE are insufficient to judge exemplification quality.

Question Answering

RankGen: Improving Text Generation with Large Ranking Models

1 code implementation19 May 2022 Kalpesh Krishna, Yapei Chang, John Wieting, Mohit Iyyer

Given an input sequence (or prefix), modern language models often assign high probabilities to output sequences that are repetitive, incoherent, or irrelevant to the prefix; as such, model-generated text also contains such artifacts.

Contrastive Learning Language Modelling +1

ChapterBreak: A Challenge Dataset for Long-Range Language Models

1 code implementation22 Apr 2022 Simeng Sun, Katherine Thai, Mohit Iyyer

While numerous architectures for long-range language models (LRLMs) have recently been proposed, a meaningful evaluation of their discourse-level language understanding capabilities has not yet followed.

RELIC: Retrieving Evidence for Literary Claims

1 code implementation ACL 2022 Katherine Thai, Yapei Chang, Kalpesh Krishna, Mohit Iyyer

Humanities scholars commonly provide evidence for claims that they make about a work of literature (e. g., a novel) in the form of quotations from the work.

Information Retrieval Semantic Similarity +1

Do Long-Range Language Models Actually Use Long-Range Context?

no code implementations EMNLP 2021 Simeng Sun, Kalpesh Krishna, Andrew Mattarella-Micke, Mohit Iyyer

Language models are generally trained on short, truncated input sequences, which limits their ability to use discourse-level information present in long-range context to improve their predictions.

The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generation

no code implementations EMNLP 2021 Marzena Karpinska, Nader Akoury, Mohit Iyyer

Recent text generation research has increasingly focused on open-ended domains such as story and poetry generation.

Text Generation

STraTA: Self-Training with Task Augmentation for Better Few-shot Learning

1 code implementation EMNLP 2021 Tu Vu, Minh-Thang Luong, Quoc V. Le, Grady Simon, Mohit Iyyer

Despite their recent successes in tackling many NLP tasks, large-scale pre-trained language models do not perform as well in few-shot settings where only a handful of training examples are available.

Few-Shot Learning Few-Shot NLI

Phrase-BERT: Improved Phrase Embeddings from BERT with an Application to Corpus Exploration

1 code implementation EMNLP 2021 Shufan Wang, Laure Thompson, Mohit Iyyer

Phrase representations derived from BERT often do not exhibit complex phrasal compositionality, as the model relies instead on lexical similarity to determine semantic relatedness.

Paraphrase Generation Topic Models

TABBIE: Pretrained Representations of Tabular Data

1 code implementation NAACL 2021 Hiroshi Iida, Dung Thai, Varun Manjunatha, Mohit Iyyer

Existing work on tabular representation learning jointly models tables and associated text using self-supervised objective functions derived from pretrained language models such as BERT.

Pretrained Language Models Representation Learning +1

IGA : An Intent-Guided Authoring Assistant

1 code implementation14 Apr 2021 Simeng Sun, Wenlong Zhao, Varun Manjunatha, Rajiv Jain, Vlad Morariu, Franck Dernoncourt, Balaji Vasan Srinivasan, Mohit Iyyer

While large-scale pretrained language models have significantly improved writing assistance functionalities such as autocomplete, more complex and controllable writing assistants have yet to be explored.

Language Modelling Pretrained Language Models

Revisiting Simple Neural Probabilistic Language Models

1 code implementation NAACL 2021 Simeng Sun, Mohit Iyyer

Recent progress in language modeling has been driven not only by advances in neural architectures, but also through hardware and optimization improvements.

Language Modelling Word Embeddings

Changing the Mind of Transformers for Topically-Controllable Language Generation

1 code implementation EACL 2021 Haw-Shiuan Chang, Jiaming Yuan, Mohit Iyyer, Andrew McCallum

Our framework consists of two components: (1) a method that produces a set of candidate topics by predicting the centers of word clusters in the possible continuations, and (2) a text generation model whose output adheres to the chosen topics.

Text Generation

Hurdles to Progress in Long-form Question Answering

1 code implementation NAACL 2021 Kalpesh Krishna, Aurko Roy, Mohit Iyyer

The task of long-form question answering (LFQA) involves retrieving documents relevant to a given question and using them to generate a paragraph-length answer.

Open-Domain Dialog Open-Domain Question Answering +1

Analyzing Gender Bias within Narrative Tropes

1 code implementation EMNLP (NLP+CSS) 2020 Dhruvil Gala, Mohammad Omar Khursheed, Hannah Lerner, Brendan O'Connor, Mohit Iyyer

Popular media reflects and reinforces societal biases through the use of tropes, which are narrative elements, such as archetypal characters and plot arcs, that occur frequently across media.

Reformulating Unsupervised Style Transfer as Paraphrase Generation

1 code implementation EMNLP 2020 Kalpesh Krishna, John Wieting, Mohit Iyyer

Modern NLP defines the task of style transfer as modifying the style of a given sentence without appreciably changing its semantics, which implies that the outputs of style transfer systems should be paraphrases of their inputs.

Paraphrase Generation Pretrained Language Models +1

Energy-Based Reranking: Improving Neural Machine Translation Using Energy-Based Models

1 code implementation ACL 2021 Sumanta Bhattacharyya, Amirmohammad Rooshenas, Subhajit Naskar, Simeng Sun, Mohit Iyyer, Andrew McCallum

To benefit from this observation, we train an energy-based model to mimic the behavior of the task measure (i. e., the energy-based model assigns lower energy to samples with higher BLEU score), which is resulted in a re-ranking algorithm based on the samples drawn from NMT: energy-based re-ranking (EBR).

Machine Translation Re-Ranking +1

Open-Retrieval Conversational Question Answering

1 code implementation22 May 2020 Chen Qu, Liu Yang, Cen Chen, Minghui Qiu, W. Bruce Croft, Mohit Iyyer

We build an end-to-end system for ORConvQA, featuring a retriever, a reranker, and a reader that are all based on Transformers.

Conversational Question Answering Conversational Search +1

Hard-Coded Gaussian Attention for Neural Machine Translation

1 code implementation ACL 2020 Weiqiu You, Simeng Sun, Mohit Iyyer

Recent work has questioned the importance of the Transformer's multi-headed attention for achieving high translation quality.

Machine Translation Translation

Exploring and Predicting Transferability across NLP Tasks

1 code implementation EMNLP 2020 Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, Mohit Iyyer

We also develop task embeddings that can be used to predict the most transferable source tasks for a given target task, and we validate their effectiveness in experiments controlled for source and target data size.

Language Modelling Part-Of-Speech Tagging +3

Thieves on Sesame Street! Model Extraction of BERT-based APIs

1 code implementation ICLR 2020 Kalpesh Krishna, Gaurav Singh Tomar, Ankur P. Parikh, Nicolas Papernot, Mohit Iyyer

We study the problem of model extraction in natural language processing, in which an adversary with only query access to a victim model attempts to reconstruct a local copy of that model.

Language Modelling Model extraction +4

Investigating Sports Commentator Bias within a Large Corpus of American Football Broadcasts

1 code implementation IJCNLP 2019 Jack Merullo, Luke Yeh, Abram Handler, Alvin Grissom II, Brendan O'Connor, Mohit Iyyer

Sports broadcasters inject drama into play-by-play commentary by building team and player narratives through subjective analyses and anecdotes.

Attentive History Selection for Conversational Question Answering

2 code implementations26 Aug 2019 Chen Qu, Liu Yang, Minghui Qiu, Yongfeng Zhang, Cen Chen, W. Bruce Croft, Mohit Iyyer

First, we propose a positional history answer embedding method to encode conversation history with position information using BERT in a natural way.

Conversational Question Answering Conversational Search +1

Encouraging Paragraph Embeddings to Remember Sentence Identity Improves Classification

1 code implementation ACL 2019 Tu Vu, Mohit Iyyer

While paragraph embedding models are remarkably effective for downstream classification tasks, what they learn and encode into a single vector remains opaque.

Classification General Classification

Generating Question-Answer Hierarchies

2 code implementations ACL 2019 Kalpesh Krishna, Mohit Iyyer

The process of knowledge acquisition can be viewed as a question-answer game between a student and a teacher in which the student typically starts by asking broad, open-ended questions before drilling down into specifics (Hintikka, 1981; Hakkarainen and Sintonen, 2002).

Language Modelling Reading Comprehension +1

Syntactically Supervised Transformers for Faster Neural Machine Translation

1 code implementation ACL 2019 Nader Akoury, Kalpesh Krishna, Mohit Iyyer

Standard decoders for neural machine translation autoregressively generate a single target token per time step, which slows inference especially for long outputs.

Machine Translation Translation

Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Auto-Encoders

1 code implementation NAACL 2019 Andrew Drozdov, Patrick Verga, Mohit Yadav, Mohit Iyyer, Andrew McCallum

We introduce the deep inside-outside recursive autoencoder (DIORA), a fully-unsupervised method for discovering syntax that simultaneously learns representations for constituents within the induced tree.

Constituency Grammar Induction

BERT with History Answer Embedding for Conversational Question Answering

1 code implementation14 May 2019 Chen Qu, Liu Yang, Minghui Qiu, W. Bruce Croft, Yongfeng Zhang, Mohit Iyyer

One of the major challenges to multi-turn conversational search is to model the conversation history to answer the current question.

Conversational Question Answering Conversational Search +1

Casting Light on Invisible Cities: Computationally Engaging with Literary Criticism

no code implementations NAACL 2019 Shufan Wang, Mohit Iyyer

Literary critics often attempt to uncover meaning in a single work of literature through careful reading and analysis.

Natural Language Processing

Quizbowl: The Case for Incremental Question Answering

no code implementations9 Apr 2019 Pedro Rodriguez, Shi Feng, Mohit Iyyer, He He, Jordan Boyd-Graber

Throughout this paper, we show that collaborations with the vibrant trivia community have contributed to the quality of our dataset, spawned new research directions, and doubled as an exciting way to engage the public with research in machine learning and natural language processing.

Decision Making Natural Language Processing +1

Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Autoencoders

3 code implementations3 Apr 2019 Andrew Drozdov, Pat Verga, Mohit Yadav, Mohit Iyyer, Andrew McCallum

We introduce deep inside-outside recursive autoencoders (DIORA), a fully-unsupervised method for discovering syntax that simultaneously learns representations for constituents within the induced tree.

Constituency Parsing

QuAC: Question Answering in Context

no code implementations EMNLP 2018 Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, Luke Zettlemoyer

We present QuAC, a dataset for Question Answering in Context that contains 14K information-seeking QA dialogs (100K questions in total).

Question Answering Reading Comprehension

A Differentiable Self-disambiguated Sense Embedding Model via Scaled Gumbel Softmax

no code implementations27 Sep 2018 Fenfei Guo, Mohit Iyyer, Leah Findlater, Jordan Boyd-Graber

We present a differentiable multi-prototype word representation model that disentangles senses of polysemous words and produces meaningful sense-specific embeddings without external resources.

Hard Attention Word Similarity

Revisiting the Importance of Encoding Logic Rules in Sentiment Classification

1 code implementation EMNLP 2018 Kalpesh Krishna, Preethi Jyothi, Mohit Iyyer

We analyze the performance of different sentiment classification models on syntactically complex inputs like A-but-B sentences.

Classification General Classification +1

QuAC : Question Answering in Context

no code implementations21 Aug 2018 Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, Luke Zettlemoyer

We present QuAC, a dataset for Question Answering in Context that contains 14K information-seeking QA dialogs (100K questions in total).

Question Answering Reading Comprehension

Inducing and Embedding Senses with Scaled Gumbel Softmax

no code implementations22 Apr 2018 Fenfei Guo, Mohit Iyyer, Jordan Boyd-Graber

Methods for learning word sense embeddings represent a single word with multiple sense-specific vectors.

Pathologies of Neural Models Make Interpretations Difficult

no code implementations EMNLP 2018 Shi Feng, Eric Wallace, Alvin Grissom II, Mohit Iyyer, Pedro Rodriguez, Jordan Boyd-Graber

In existing interpretation methods for NLP, a word's importance is determined by either input perturbation---measuring the decrease in model confidence when that word is removed---or by the gradient with respect to that word.

Adversarial Example Generation with Syntactically Controlled Paraphrase Networks

2 code implementations NAACL 2018 Mohit Iyyer, John Wieting, Kevin Gimpel, Luke Zettlemoyer

We propose syntactically controlled paraphrase networks (SCPNs) and use them to generate adversarial examples.

Deep contextualized word representations

42 code implementations NAACL 2018 Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, Luke Zettlemoyer

We introduce a new type of deep contextualized word representation that models both (1) complex characteristics of word use (e. g., syntax and semantics), and (2) how these uses vary across linguistic contexts (i. e., to model polysemy).

Ranked #2 on Citation Intent Classification on ACL-ARC (using extra training data)

Citation Intent Classification Conversational Response Selection +7

Search-based Neural Structured Learning for Sequential Question Answering

no code implementations ACL 2017 Mohit Iyyer, Wen-tau Yih, Ming-Wei Chang

Recent work in semantic parsing for question answering has focused on long and complicated questions, many of which would seem unnatural if asked in a normal conversation between two humans.

Question Answering Semantic Parsing

The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives

1 code implementation CVPR 2017 Mohit Iyyer, Varun Manjunatha, Anupam Guha, Yogarshi Vyas, Jordan Boyd-Graber, Hal Daumé III, Larry Davis

While computers can now describe what is explicitly depicted in natural images, in this paper we examine whether they can understand the closure-driven narratives conveyed by stylized artwork and dialogue in comic book panels.

Answering Complicated Question Intents Expressed in Decomposed Question Sequences

no code implementations4 Nov 2016 Mohit Iyyer, Wen-tau Yih, Ming-Wei Chang

Recent work in semantic parsing for question answering has focused on long and complicated questions, many of which would seem unnatural if asked in a normal conversation between two humans.

Question Answering Semantic Parsing

Cannot find the paper you are looking for? You can Submit a new open access paper.