Search Results for author: Hal Daumé III

Found 61 papers, 24 papers with code

Distantly-Supervised Dense Retrieval Enables Open-Domain Question Answering without Evidence Annotation

1 code implementation EMNLP 2021 Chen Zhao, Chenyan Xiong, Jordan Boyd-Graber, Hal Daumé III

This paper investigates whether models can learn to find evidence from a large corpus, with only distant supervision from answer labels for model training, thereby generating no additional annotation cost.

Open-Domain Question Answering Retrieval

Toward Gender-Inclusive Coreference Resolution: An Analysis of Gender and Bias Throughout the Machine Learning Lifecycle*

no code implementations CL (ACL) 2021 Yang Trista Cao, Hal Daumé III

Such inferences raise the risk of systematic biases in coreference resolution systems, including biases that can harm binary and non-binary trans and cis stakeholders.

coreference-resolution Sociology

TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learning

no code implementations22 Jun 2023 Ruijie Zheng, Xiyao Wang, Yanchao Sun, Shuang Ma, Jieyu Zhao, Huazhe Xu, Hal Daumé III, Furong Huang

Despite recent progress in reinforcement learning (RL) from raw pixel data, sample inefficiency continues to present a substantial obstacle.

Continuous Control Contrastive Learning +3

Evaluating the Social Impact of Generative AI Systems in Systems and Society

no code implementations9 Jun 2023 Irene Solaiman, Zeerak Talat, William Agnew, Lama Ahmad, Dylan Baker, Su Lin Blodgett, Hal Daumé III, Jesse Dodge, Ellie Evans, Sara Hooker, Yacine Jernite, Alexandra Sasha Luccioni, Alberto Lusoli, Margaret Mitchell, Jessica Newman, Marie-Therese Png, Andrew Strait, Apostol Vassilev

We move toward a standard approach in evaluating a generative AI system for any modality, in two overarching categories: what is able to be evaluated in a base system that has no predetermined application and what is able to be evaluated in society.

What Else Do I Need to Know? The Effect of Background Information on Users' Reliance on AI Systems

no code implementations23 May 2023 Navita Goyal, Eleftheria Briakou, Amanda Liu, Connor Baumler, Claire Bonial, Jeffrey Micher, Clare R. Voss, Marine Carpuat, Hal Daumé III

This leads to a mismatch between the information that these models access to derive the answer and the information available to the user consuming the AI predictions to assess the AI predicted answer.

It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performance

no code implementations15 May 2023 Arjun Subramonian, Xingdi Yuan, Hal Daumé III, Su Lin Blodgett

Progress in NLP is increasingly measured through benchmarks; hence, contextualizing progress requires understanding when and why practitioners may disagree about the validity of benchmarks.

coreference-resolution Question Answering

ASL Citizen: A Community-Sourced Dataset for Advancing Isolated Sign Language Recognition

no code implementations12 Apr 2023 Aashaka Desai, Lauren Berger, Fyodor O. Minakov, Vanessa Milan, Chinmay Singh, Kriston Pumphrey, Richard E. Ladner, Hal Daumé III, Alex X. Lu, Naomi Caselli, Danielle Bragg

We propose that this dataset be used for sign language dictionary retrieval for American Sign Language (ASL), where a user demonstrates a sign to their webcam to retrieve matching signs from a dictionary.

Retrieval Sign Language Recognition

Define, Evaluate, and Improve Task-Oriented Cognitive Capabilities for Instruction Generation Models

no code implementations21 Dec 2022 Lingjun Zhao, Khanh Nguyen, Hal Daumé III

Recent work studies the cognitive capabilities of language models through psychological tests designed for humans.

Language Modelling

How do Authors' Perceptions of their Papers Compare with Co-authors' Perceptions and Peer-review Decisions?

no code implementations22 Nov 2022 Charvi Rastogi, Ivan Stelmakh, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, Jennifer Wortman Vaughan, Zhenyu Xue, Hal Daumé III, Emma Pierson, Nihar B. Shah

In a top-tier computer science conference (NeurIPS 2021) with more than 23, 000 submitting authors and 9, 000 submitted papers, we survey the authors on three questions: (i) their predicted probability of acceptance for each of their papers, (ii) their perceived ranking of their own papers based on scientific contribution, and (iii) the change in their perception about their own papers after seeing the reviews.

What's Different between Visual Question Answering for Machine "Understanding" Versus for Accessibility?

1 code implementation26 Oct 2022 Yang Trista Cao, Kyle Seelman, Kyungjun Lee, Hal Daumé III

We aim to answer this question by evaluating discrepancies between machine "understanding" datasets (VQA-v2) and accessibility datasets (VizWiz) by evaluating a variety of VQA models.

Benchmarking Question Answering +1

Theory-Grounded Measurement of U.S. Social Stereotypes in English Language Models

1 code implementation NAACL 2022 Yang Trista Cao, Anna Sotnikova, Hal Daumé III, Rachel Rudinger, Linda Zou

NLP models trained on text have been shown to reproduce human stereotypes, which can magnify harms to marginalized groups when systems are deployed at scale.

A Framework for Learning to Request Rich and Contextually Useful Information from Humans

no code implementations14 Oct 2021 Khanh Nguyen, Yonatan Bisk, Hal Daumé III

We show that the agent can take advantage of different types of information depending on the context, and analyze the benefits and challenges of learning the assistance-requesting policy when the assistant can recursively decompose tasks into subtasks.

Decision Making Hierarchical Reinforcement Learning

Learning When and What to Ask: a Hierarchical Reinforcement Learning Framework

no code implementations29 Sep 2021 Khanh Xuan Nguyen, Yonatan Bisk, Hal Daumé III

Results on a simulated human-assisted navigation problem demonstrate the effectiveness of our framework: aided with an interaction policy learned by our method, a navigation policy achieves up to a 7× improvement in task success rate compared to performing tasks only by itself.

Hierarchical Reinforcement Learning reinforcement-learning +1

A Large Scale Randomized Controlled Trial on Herding in Peer-Review Discussions

no code implementations30 Nov 2020 Ivan Stelmakh, Charvi Rastogi, Nihar B. Shah, Aarti Singh, Hal Daumé III

Peer review is the backbone of academia and humans constitute a cornerstone of this process, being responsible for reviewing papers and making the final acceptance/rejection decisions.

Decision Making

Prior and Prejudice: The Novice Reviewers' Bias against Resubmissions in Conference Peer Review

no code implementations30 Nov 2020 Ivan Stelmakh, Nihar B. Shah, Aarti Singh, Hal Daumé III

Modern machine learning and computer science conferences are experiencing a surge in the number of submissions that challenges the quality of peer review as the number of competent reviewers is growing at a much slower rate.

BIG-bench Machine Learning

A Novice-Reviewer Experiment to Address Scarcity of Qualified Reviewers in Large Conferences

no code implementations30 Nov 2020 Ivan Stelmakh, Nihar B. Shah, Aarti Singh, Hal Daumé III

Conference peer review constitutes a human-computation process whose importance cannot be overstated: not only it identifies the best submissions for acceptance, but, ultimately, it impacts the future of the whole research area by promoting some ideas and restraining others.

Language (Technology) is Power: A Critical Survey of "Bias" in NLP

1 code implementation28 May 2020 Su Lin Blodgett, Solon Barocas, Hal Daumé III, Hanna Wallach

We survey 146 papers analyzing "bias" in NLP systems, finding that their motivations are often vague, inconsistent, and lacking in normative reasoning, despite the fact that analyzing "bias" is an inherently normative process.

Operationalizing the Legal Principle of Data Minimization for Personalization

no code implementations28 May 2020 Asia J. Biega, Peter Potash, Hal Daumé III, Fernando Diaz, Michèle Finck

Article 5(1)(c) of the European Union's General Data Protection Regulation (GDPR) requires that "personal data shall be [...] adequate, relevant, and limited to what is necessary in relation to the purposes for which they are processed (`data minimisation')".

Recommendation Systems

Active Imitation Learning with Noisy Guidance

1 code implementation ACL 2020 Kianté Brantley, Amr Sharaf, Hal Daumé III

Imitation learning algorithms provide state-of-the-art results on many structured prediction tasks by learning near-optimal search policies.

Active Learning Imitation Learning +1

Meta-Learning for Few-Shot NMT Adaptation

no code implementations WS 2020 Amr Sharaf, Hany Hassan, Hal Daumé III

We frame the adaptation of NMT systems as a meta-learning problem, where we learn to adapt to new unseen domains based on simulated offline meta-training domain adaptation tasks.

Domain Adaptation Machine Translation +3

Toward Gender-Inclusive Coreference Resolution

1 code implementation ACL 2020 Yang Trista Cao, Hal Daumé III

Correctly resolving textual mentions of people fundamentally entails making inferences about those people.

coreference-resolution Sociology

Weight of Evidence as a Basis for Human-Oriented Explanations

1 code implementation29 Oct 2019 David Alvarez-Melis, Hal Daumé III, Jennifer Wortman Vaughan, Hanna Wallach

Interpretability is an elusive but highly sought-after characteristic of modern machine learning methods.


Learning Effective Exploration Strategies For Contextual Bandits

no code implementations25 Sep 2019 Amr Sharaf, Hal Daumé III

We develop a meta-learning algorithm, MELEE, that learns an exploration policy based on simulated, synthetic contextual bandit tasks.

Imitation Learning Learning-To-Rank +2

Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learning

1 code implementation IJCNLP 2019 Khanh Nguyen, Hal Daumé III

An agent solving tasks in a HANNA environment can leverage simulated human assistants, called ANNA (Automatic Natural Navigation Assistants), which, upon request, provide natural language and visual instructions to direct the agent towards the goals.

Imitation Learning Visual Navigation

Answer-based Adversarial Training for Generating Clarification Questions

1 code implementation NAACL 2019 Sudha Rao, Hal Daumé III

We present an approach for generating clarification questions with the goal of eliciting new information that would make the given textual context more complete.

Retrieval Specificity

Non-Monotonic Sequential Text Generation

1 code implementation WS 2019 Sean Welleck, Kianté Brantley, Hal Daumé III, Kyunghyun Cho

Standard sequential generation methods assume a pre-specified generation order, such as text generation methods which generate words from left to right.

Imitation Learning Text Generation

Meta-Learning for Contextual Bandit Exploration

no code implementations ICLR 2019 Amr Sharaf, Hal Daumé III

We describe MELEE, a meta-learning algorithm for learning a good exploration policy in the interactive contextual bandit setting.

Imitation Learning Meta-Learning

Warm-starting Contextual Bandits: Robustly Combining Supervised and Bandit Feedback

1 code implementation2 Jan 2019 Chicheng Zhang, Alekh Agarwal, Hal Daumé III, John Langford, Sahand N. Negahban

We investigate the feasibility of learning from a mix of both fully-labeled supervised data and contextual bandit data.

Multi-Armed Bandits

Improving fairness in machine learning systems: What do industry practitioners need?

no code implementations13 Dec 2018 Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé III, Miro Dudík, Hanna Wallach

The potential for machine learning (ML) systems to amplify social inequities and unfairness is receiving increasing popular and academic attention.

BIG-bench Machine Learning Fairness

Contextual Memory Trees

no code implementations17 Jul 2018 Wen Sun, Alina Beygelzimer, Hal Daumé III, John Langford, Paul Mineiro

We design and study a Contextual Memory Tree (CMT), a learning memory controller that inserts new memories into an experience store of unbounded size.

General Classification Image Captioning +2

Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information

1 code implementation ACL 2018 Sudha Rao, Hal Daumé III

Inquiry is fundamental to communication, and machines cannot effectively collaborate with humans unless they can ask questions.

Datasheets for Datasets

20 code implementations23 Mar 2018 Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, Kate Crawford

The machine learning community currently has no standardized process for documenting datasets, which can lead to severe consequences in high-stakes domains.

BIG-bench Machine Learning

Technical Report: When Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning Attacks

no code implementations19 Mar 2018 Octavian Suciu, Radu Mărginean, Yiğitcan Kaya, Hal Daumé III, Tudor Dumitraş

Our model allows us to consider a wide range of weaker adversaries who have limited control and incomplete knowledge of the features, learning algorithms and training instances utilized.

BIG-bench Machine Learning

Residual Loss Prediction: Reinforcement Learning With No Incremental Feedback

1 code implementation ICLR 2018 Hal Daumé III, John Langford, Amr Sharaf

We consider reinforcement learning and bandit structured prediction problems with very sparse loss feedback: only at the end of an episode.

Multi-Armed Bandits reinforcement-learning +2

Towards Linguistically Generalizable NLP Systems: A Workshop and Shared Task

no code implementations WS 2017 Allyson Ettinger, Sudha Rao, Hal Daumé III, Emily M. Bender

This paper presents a summary of the first Workshop on Building Linguistically Generalizable Natural Language Processing Systems, and the associated Build It Break It, The Language Edition shared task.

Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback

1 code implementation EMNLP 2017 Khanh Nguyen, Hal Daumé III, Jordan Boyd-Graber

Machine translation is a natural candidate problem for reinforcement learning from human feedback: users provide quick, dirty ratings on candidate translations to guide a system to improve.

Machine Translation reinforcement-learning +2

The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives

2 code implementations CVPR 2017 Mohit Iyyer, Varun Manjunatha, Anupam Guha, Yogarshi Vyas, Jordan Boyd-Graber, Hal Daumé III, Larry Davis

While computers can now describe what is explicitly depicted in natural images, in this paper we examine whether they can understand the closure-driven narratives conveyed by stylized artwork and dialogue in comic book panels.

Opponent Modeling in Deep Reinforcement Learning

1 code implementation18 Sep 2016 He He, Jordan Boyd-Graber, Kevin Kwok, Hal Daumé III

Opponent modeling is necessary in multi-agent settings where secondary agents with competing goals also adapt their strategies, yet it remains challenging because strategies interact with each other and change.

reinforcement-learning Reinforcement Learning (RL)

On Correcting Inputs: Inverse Optimization for Online Structured Prediction

no code implementations12 Oct 2015 Hal Daumé III, Samir Khuller, Manish Purohit, Gregory Sanders

Algorithm designers typically assume that the input data is correct, and then proceed to find "optimal" or "sub-optimal" solutions using this input data.

Structured Prediction

Learning to Search for Dependencies

no code implementations18 Mar 2015 Kai-Wei Chang, He He, Hal Daumé III, John Langford

We demonstrate that a dependency parser can be built using a credit assignment compiler which removes the burden of worrying about low-level machine learning details from the parser implementation.

BIG-bench Machine Learning

Learning Reductions that Really Work

no code implementations9 Feb 2015 Alina Beygelzimer, Hal Daumé III, John Langford, Paul Mineiro

We provide a summary of the mathematical and computational techniques that have enabled learning reductions to effectively address a wide class of problems, and show that this approach to solving machine learning problems can be broadly useful.

BIG-bench Machine Learning

Learning to Search Better Than Your Teacher

no code implementations8 Feb 2015 Kai-Wei Chang, Akshay Krishnamurthy, Alekh Agarwal, Hal Daumé III, John Langford

Methods for learning to search for structured prediction typically imitate a reference policy, with existing theoretical guarantees demonstrating low regret compared to that reference.

Multi-Armed Bandits Structured Prediction

A Credit Assignment Compiler for Joint Prediction

no code implementations NeurIPS 2016 Kai-Wei Chang, He He, Hal Daumé III, John Langford, Stephane Ross

Many machine learning applications involve jointly predicting multiple mutually dependent output variables.

Frustratingly Easy Domain Adaptation

no code implementations10 Jul 2009 Hal Daumé III

We describe an approach to domain adaptation that is appropriate exactly in the case when one has enough ``target'' data to do slightly better than just using only ``source'' data.

Domain Adaptation

A Bayesian Model for Discovering Typological Implications

1 code implementation4 Jul 2009 Hal Daumé III, Lyle Campbell

A standard form of analysis for linguistic typology is the universal implication.

Search-based Structured Prediction

no code implementations4 Jul 2009 Hal Daumé III, John Langford, Daniel Marcu

We present Searn, an algorithm for integrating search and learning to solve complex structured prediction problems such as those that occur in natural language, speech, computational biology, and vision.

General Classification Structured Prediction

Cannot find the paper you are looking for? You can Submit a new open access paper.