Search Results for author: Jessica Hullman

Found 11 papers, 2 papers with code

A Statistical Framework for Measuring AI Reliance

no code implementations27 Jan 2024 Ziyang Guo, Yifan Wu, Jason Hartline, Jessica Hullman

We argue that the current definition of appropriate reliance used in such research lacks formal statistical grounding and can lead to contradictions.

Decision Making

Decision Theoretic Foundations for Experiments Evaluating Human Decisions

no code implementations25 Jan 2024 Jessica Hullman, Alex Kale, Jason Hartline

We argue that to attribute loss in human performance to forms of bias, an experiment must provide participants with the information that a rational agent would need to identify the normative decision.

Attribute Data Visualization +1

Evaluating the Utility of Conformal Prediction Sets for AI-Advised Image Labeling

no code implementations16 Jan 2024 Dongping Zhang, Angelos Chatzimparmpas, Negar Kamali, Jessica Hullman

As deep neural networks are more commonly deployed in high-stakes domains, their black-box nature makes uncertainty quantification challenging.

Conformal Prediction Decision Making +2

Pre-registration for Predictive Modeling

no code implementations30 Nov 2023 Jake M. Hofman, Angelos Chatzimparmpas, Amit Sharma, Duncan J. Watts, Jessica Hullman

Amid rising concerns of reproducibility and generalizability in predictive modeling, we explore the possibility and potential benefits of introducing pre-registration to the field.

Decision Making

Artificial Intelligence and Aesthetic Judgment

no code implementations21 Aug 2023 Jessica Hullman, Ari Holtzman, Andrew Gelman

In this essay, we focus on an unresolved tension when we bring this dilemma to bear in the context of generative AI: are we looking for proof that generated media reflects something about the conditions that created it or some eternal human essence?

Causal Inference

Are We Closing the Loop Yet? Gaps in the Generalizability of VIS4ML Research

no code implementations10 Aug 2023 Hariharan Subramonyam, Jessica Hullman

Visualization for machine learning (VIS4ML) research aims to help experts apply their prior knowledge to develop, understand, and improve the performance of machine learning models.

The worst of both worlds: A comparative analysis of errors in learning from data in psychology and machine learning

no code implementations12 Mar 2022 Jessica Hullman, Sayash Kapoor, Priyanka Nanayakkara, Andrew Gelman, Arvind Narayanan

We conclude by discussing risks that arise when sources of errors are misdiagnosed and the need to acknowledge the role of human inductive biases in learning and reform.

Causal Inference

Visual Reasoning Strategies for Effect Size Judgments and Decisions

2 code implementations28 Jul 2020 Alex Kale, Matthew Kay, Jessica Hullman

We also see that visualization designs that support the least biased effect size estimation do not support the best decision-making, suggesting that a chart user's sense of effect size may not necessarily be identical when they use the same information for different tasks.

Human-Computer Interaction

Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs

1 code implementation23 Apr 2020 Sungsoo Ray Hong, Jessica Hullman, Enrico Bertini

As the use of machine learning (ML) models in product development and data-driven decision-making processes became pervasive in many domains, people's focus on building a well-performing model has increasingly shifted to understanding how their model works.

Decision Making

Cannot find the paper you are looking for? You can Submit a new open access paper.