Search Results for author: Jordan Boyd-Graber

Found 96 papers, 24 papers with code

Eliciting Bias in Question Answering Models through Ambiguity

1 code implementation EMNLP (MRQA) 2021 Andrew Mao, Naveen Raman, Matthew Shu, Eric Li, Franklin Yang, Jordan Boyd-Graber

We develop two sets of questions for closed and open domain questions respectively, which use ambiguous questions to probe QA models for bias.

Question Answering

Distantly-Supervised Dense Retrieval Enables Open-Domain Question Answering without Evidence Annotation

1 code implementation EMNLP 2021 Chen Zhao, Chenyan Xiong, Jordan Boyd-Graber, Hal Daumé III

This paper investigates whether models can learn to find evidence from a large corpus, with only distant supervision from answer labels for model training, thereby generating no additional annotation cost.

Open-Domain Question Answering

Adapting Entities across Languages and Cultures

no code implementations Findings (EMNLP) 2021 Denis Peskov, Viktor Hangya, Jordan Boyd-Graber, Alexander Fraser

He is associated with founding a company in the United States, so perhaps the German founder Carl Benz could stand in for Gates in those contexts.

Machine Translation Question Answering +1

Evaluation Paradigms in Question Answering

no code implementations EMNLP 2021 Pedro Rodriguez, Jordan Boyd-Graber

Question answering (QA) primarily descends from two branches of research: (1) Alan Turing’s investigation of machine intelligence at Manchester University and (2) Cyril Cleverdon’s comparison of library card catalog indices at Cranfield University.

Question Answering

What’s in a Name? Answer Equivalence For Open-Domain Question Answering

1 code implementation EMNLP 2021 Chenglei Si, Chen Zhao, Jordan Boyd-Graber

We incorporate answers for two settings: evaluation with additional answers and model training with equivalent answers.

Open-Domain Question Answering

Automatic Song Translation for Tonal Languages

no code implementations Findings (ACL) 2022 Fenfei Guo, Chen Zhang, Zhirui Zhang, Qixin He, Kejun Zhang, Jun Xie, Jordan Boyd-Graber

This paper develops automatic song translation (AST) for tonal languages and addresses the unique challenge of aligning words' tones with melody of a song in addition to conveying the original meaning.

Translation

What's in a Name? Answer Equivalence For Open-Domain Question Answering

1 code implementation11 Sep 2021 Chenglei Si, Chen Zhao, Jordan Boyd-Graber

We incorporate answers for two settings: evaluation with additional answers and model training with equivalent answers.

Open-Domain Question Answering

Evaluation Examples are not Equally Informative: How should that change NLP Leaderboards?

1 code implementation ACL 2021 Pedro Rodriguez, Joe Barrow, Alexander Miserlis Hoyle, John P. Lalor, Robin Jia, Jordan Boyd-Graber

While leaderboards are a straightforward ranking of NLP models, this simplicity can mask nuances in evaluation items (examples) and subjects (NLP models).

Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherence

1 code implementation NeurIPS 2021 Alexander Hoyle, Pranav Goel, Denis Peskov, Andrew Hian-Cheong, Jordan Boyd-Graber, Philip Resnik

To address the standardization gap, we systematically evaluate a dominant classical model and two state-of-the-art neural models on two commonly used datasets.

Topic Models

Fool Me Twice: Entailment from Wikipedia Gamification

1 code implementation NAACL 2021 Julian Martin Eisenschlos, Bhuwan Dhingra, Jannis Bulian, Benjamin Börschinger, Jordan Boyd-Graber

We release FoolMeTwice (FM2 for short), a large dataset of challenging entailment pairs collected through a fun multi-player game.

Complex Factoid Question Answering with a Free-Text Knowledge Graph

no code implementations23 Mar 2021 Chen Zhao, Chenyan Xiong, Xin Qian, Jordan Boyd-Graber

DELFT's advantage comes from both the high coverage of its free-text knowledge graph-more than double that of dbpedia relations-and the novel graph neural network which reasons on the rich but noisy free-text evidence.

Graph Question Answering Question Answering +1

An Attentive Recurrent Model for Incremental Prediction of Sentence-final Verbs

no code implementations Findings of the Association for Computational Linguistics 2020 Wenyan Li, Alvin Grissom II, Jordan Boyd-Graber

Verb prediction is important for understanding human processing of verb-final languages, with practical applications to real-time simultaneous interpretation from verb-final to verb-medial languages.

It Takes Two to Lie: One to Lie, and One to Listen

no code implementations ACL 2020 Denis Peskov, Benny Cheng, Ahmed Elgohary, Joe Barrow, Cristian Danescu-Niculescu-Mizil, Jordan Boyd-Graber

Trust is implicit in many online text conversations{---}striking up new friendships, or asking for tech support.

Meta Answering for Machine Reading

no code implementations11 Nov 2019 Benjamin Borschinger, Jordan Boyd-Graber, Christian Buck, Jannis Bulian, Massimiliano Ciaramita, Michelle Chen Huebscher, Wojciech Gajewski, Yannic Kilcher, Rodrigo Nogueira, Lierni Sestorain Saralegu

We investigate a framework for machine reading, inspired by real world information-seeking problems, where a meta question answering system interacts with a black box environment.

Question Answering Reading Comprehension

Interactive Refinement of Cross-Lingual Word Embeddings

1 code implementation EMNLP 2020 Michelle Yuan, Mozhi Zhang, Benjamin Van Durme, Leah Findlater, Jordan Boyd-Graber

Cross-lingual word embeddings transfer knowledge between languages: models trained on high-resource languages can predict in low-resource languages.

Active Learning Cross-Lingual Word Embeddings +3

How Pre-trained Word Representations Capture Commonsense Physical Comparisons

no code implementations WS 2019 Pranav Goel, Shi Feng, Jordan Boyd-Graber

One type of common sense is how two objects compare on physical properties such as size and weight: e. g., {`}is a house bigger than a person?{'}.

Common Sense Reasoning

Can You Unpack That? Learning to Rewrite Questions-in-Context

no code implementations IJCNLP 2019 Ahmed Elgohary, Denis Peskov, Jordan Boyd-Graber

Question answering is an AI-complete problem, but existing datasets lack key elements of language understanding such as coreference and ellipsis resolution.

Question Answering

What Question Answering can Learn from Trivia Nerds

no code implementations ACL 2020 Jordan Boyd-Graber, Benjamin Börschinger

In addition to the traditional task of getting machines to answer questions, a major research question in question answering is to create interesting, challenging questions that can help systems learn how to answer questions and also reveal which systems are the best at answering questions.

Question Answering

Mitigating Noisy Inputs for Question Answering

no code implementations8 Aug 2019 Denis Peskov, Joe Barrow, Pedro Rodriguez, Graham Neubig, Jordan Boyd-Graber

We investigate and mitigate the effects of noise from Automatic Speech Recognition systems on two factoid Question Answering (QA) tasks.

Automatic Speech Recognition Machine Translation +3

Are Girls Neko or Sh\=ojo? Cross-Lingual Alignment of Non-Isomorphic Embeddings with Iterative Normalization

no code implementations ACL 2019 Mozhi Zhang, Keyulu Xu, Ken-ichi Kawarabayashi, Stefanie Jegelka, Jordan Boyd-Graber

Cross-lingual word embeddings (CLWE) underlie many multilingual natural language processing systems, often through orthogonal transformations of pre-trained monolingual embeddings.

Cross-Lingual Word Embeddings Translation +2

Are Girls Neko or Shōjo? Cross-Lingual Alignment of Non-Isomorphic Embeddings with Iterative Normalization

1 code implementation4 Jun 2019 Mozhi Zhang, Keyulu Xu, Ken-ichi Kawarabayashi, Stefanie Jegelka, Jordan Boyd-Graber

Cross-lingual word embeddings (CLWE) underlie many multilingual natural language processing systems, often through orthogonal transformations of pre-trained monolingual embeddings.

Cross-Lingual Word Embeddings Translation +2

Why Didn't You Listen to Me? Comparing User Control of Human-in-the-Loop Topic Models

no code implementations ACL 2019 Varun Kumar, Alison Smith-Renner, Leah Findlater, Kevin Seppi, Jordan Boyd-Graber

To address the lack of comparative evaluation of Human-in-the-Loop Topic Modeling (HLTM) systems, we implement and evaluate three contrasting HLTM modeling approaches using simulation experiments.

Topic Models

Automatic Evaluation of Local Topic Quality

no code implementations ACL 2019 Jeffrey Lund, Piper Armstrong, Wilson Fearn, Stephen Cowley, Courtni Byun, Jordan Boyd-Graber, Kevin Seppi

Topic models are typically evaluated with respect to the global topic distributions that they generate, using metrics such as coherence, but without regard to local (token-level) topic assignments.

Topic Models

Misleading Failures of Partial-input Baselines

no code implementations ACL 2019 Shi Feng, Eric Wallace, Jordan Boyd-Graber

Recent work establishes dataset difficulty and removes annotation artifacts via partial-input baselines (e. g., hypothesis-only models for SNLI or question-only models for VQA).

Natural Language Inference Visual Question Answering +1

Quizbowl: The Case for Incremental Question Answering

no code implementations9 Apr 2019 Pedro Rodriguez, Shi Feng, Mohit Iyyer, He He, Jordan Boyd-Graber

Throughout this paper, we show that collaborations with the vibrant trivia community have contributed to the quality of our dataset, spawned new research directions, and doubled as an exciting way to engage the public with research in machine learning and natural language processing.

Decision Making Question Answering

What can AI do for me: Evaluating Machine Learning Interpretations in Cooperative Play

no code implementations23 Oct 2018 Shi Feng, Jordan Boyd-Graber

Machine learning is an important tool for decision making, but its ethical and responsible application requires rigorous vetting of its interpretability and utility: an understudied problem, particularly for natural language processing models.

Decision Making Question Answering

A dataset and baselines for sequential open-domain question answering

no code implementations EMNLP 2018 Ahmed Elgohary, Chen Zhao, Jordan Boyd-Graber

Previous work on question-answering systems mainly focuses on answering individual questions, assuming they are independent and devoid of context.

Information Retrieval Open-Domain Question Answering +1

A Differentiable Self-disambiguated Sense Embedding Model via Scaled Gumbel Softmax

no code implementations27 Sep 2018 Fenfei Guo, Mohit Iyyer, Leah Findlater, Jordan Boyd-Graber

We present a differentiable multi-prototype word representation model that disentangles senses of polysemous words and produces meaningful sense-specific embeddings without external resources.

Hard Attention Word Similarity

Automatic Estimation of Simultaneous Interpreter Performance

1 code implementation ACL 2018 Craig Stewart, Nikolai Vogler, Junjie Hu, Jordan Boyd-Graber, Graham Neubig

Simultaneous interpretation, translation of the spoken word in real-time, is both highly challenging and physically demanding.

Machine Translation Translation

Lessons from the Bible on Modern Topics: Low-Resource Multilingual Topic Model Evaluation

no code implementations NAACL 2018 Shudong Hao, Jordan Boyd-Graber, Michael J. Paul

Multilingual topic models enable document analysis across languages through coherent multilingual summaries of the data.

Topic Models

Inducing and Embedding Senses with Scaled Gumbel Softmax

no code implementations22 Apr 2018 Fenfei Guo, Mohit Iyyer, Jordan Boyd-Graber

Methods for learning word sense embeddings represent a single word with multiple sense-specific vectors.

Pathologies of Neural Models Make Interpretations Difficult

no code implementations EMNLP 2018 Shi Feng, Eric Wallace, Alvin Grissom II, Mohit Iyyer, Pedro Rodriguez, Jordan Boyd-Graber

In existing interpretation methods for NLP, a word's importance is determined by either input perturbation---measuring the decrease in model confidence when that word is removed---or by the gradient with respect to that word.

Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback

1 code implementation EMNLP 2017 Khanh Nguyen, Hal Daumé III, Jordan Boyd-Graber

Machine translation is a natural candidate problem for reinforcement learning from human feedback: users provide quick, dirty ratings on candidate translations to guide a system to improve.

Machine Translation reinforcement-learning +1

Tandem Anchoring: a Multiword Anchor Approach for Interactive Topic Modeling

no code implementations ACL 2017 Jeffrey Lund, Connor Cook, Kevin Seppi, Jordan Boyd-Graber

We propose combinations of words as anchors, going beyond existing single word anchor algorithms{---}an approach we call {``}Tandem Anchors{''}.

Document Classification Information Retrieval +2

The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives

1 code implementation CVPR 2017 Mohit Iyyer, Varun Manjunatha, Anupam Guha, Yogarshi Vyas, Jordan Boyd-Graber, Hal Daumé III, Larry Davis

While computers can now describe what is explicitly depicted in natural images, in this paper we examine whether they can understand the closure-driven narratives conveyed by stylized artwork and dialogue in comic book panels.

Opponent Modeling in Deep Reinforcement Learning

1 code implementation18 Sep 2016 He He, Jordan Boyd-Graber, Kevin Kwok, Hal Daumé III

Opponent modeling is necessary in multi-agent settings where secondary agents with competing goals also adapt their strategies, yet it remains challenging because strategies interact with each other and change.

reinforcement-learning

Online Adaptor Grammars with Hybrid Inference

no code implementations TACL 2014 Ke Zhai, Jordan Boyd-Graber, Shay B. Cohen

Adaptor grammars are a flexible, powerful formalism for defining nonparametric, unsupervised models of grammar productions.

Topic Models Variational Inference

Cannot find the paper you are looking for? You can Submit a new open access paper.