Search Results for author:
Found 31 papers, 9 papers with code
Open Datasheets: Machine-readable Documentation for Open Datasets and Responsible AI Assessments
This paper introduces a no-code, machine-readable documentation framework for open datasets, with a focus on responsible AI (RAI) considerations.
Has the Machine Learning Review Process Become More Arbitrary as the Field Has Grown? The NeurIPS 2021 Consistency Experiment
We present the NeurIPS 2021 consistency experiment, a larger-scale variant of the 2014 NeurIPS experiment in which 10% of conference submissions were reviewed by two independent committees to quantify the randomness in the review process.
AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap
It is paramount to pursue new approaches to provide transparency for LLMs, and years of research at the intersection of AI and human-computer interaction (HCI) highlight that we must do so with a human-centered perspective: Transparency is fundamentally about supporting appropriate human understanding, and this understanding is sought by different stakeholders with different goals in different contexts.
GAM Coach: Towards Interactive and User-centered Algorithmic Recourse
Machine learning (ML) recourse techniques are increasingly used in high-stakes domains, providing end users with actions to alter ML predictions, but they assume ML developers understand what input variables can be changed.
Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experience
To address this problem, we bridge the literature on AI design and AI transparency to explore whether and how frameworks for transparent model reporting can support design ideation with pre-trained models.
Generation Probabilities Are Not Enough: Exploring the Effectiveness of Uncertainty Highlighting in AI-Powered Code Completions
Through a mixed-methods study with 30 programmers, we compare three conditions: providing the AI system's code completion alone, highlighting tokens with the lowest likelihood of being generated by the underlying generative model, and highlighting tokens with the highest predicted likelihood of being edited by a programmer.
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
AI explanations are often mentioned as a way to improve human-AI decision-making, but empirical studies have not found consistent evidence of explanations' effectiveness and, on the contrary, suggest that they can increase overreliance when the AI system is wrong.
How do Authors' Perceptions of their Papers Compare with Co-authors' Perceptions and Peer-review Decisions?
In a top-tier computer science conference (NeurIPS 2021) with more than 23, 000 submitting authors and 9, 000 submitted papers, we survey the authors on three questions: (i) their predicted probability of acceptance for each of their papers, (ii) their perceived ranking of their own papers based on scientific contribution, and (iii) the change in their perception about their own papers after seeing the reviews.
Interpretable Distribution Shift Detection using Optimal Transport
We propose a method to identify and characterize distribution shifts in classification datasets based on optimal transport.
Interpretability, Then What? Editing Machine Learning Models to Reflect Human Knowledge and Values
Machine learning (ML) interpretability techniques can reveal undesirable patterns in data that models exploit to make predictions--potentially causing harms once deployed.
Understanding Machine Learning Practitioners' Data Documentation Perceptions, Needs, Challenges, and Desiderata
Despite the fact that data documentation frameworks are often motivated from the perspective of responsible AI, participants did not make the connection between the questions that they were asked to answer and their responsible AI implications.
REAL ML: Recognizing, Exploring, and Articulating Limitations of Machine Learning Research
Transparency around limitations can improve the scientific rigor of research, help ensure appropriate interpretation of research findings, and make research claims more credible.
Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support
Various tools and practices have been developed to support practitioners in identifying, assessing, and mitigating fairness-related harms caused by AI systems.
GAM Changer: Editing Generalized Additive Models with Interactive Visualization
Recent strides in interpretable machine learning (ML) research reveal that models exploit undesirable patterns in the data to make predictions, which potentially causes harms in deployment.
From Human Explanation to Model Interpretability: A Framework Based on Weight of Evidence
We take inspiration from the study of human explanation to inform the design and evaluation of interpretability methods in machine learning.
BIG-bench Machine Learning
Interpretable Machine Learning
+1
Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs
Disaggregated evaluations of AI systems, in which system performance is assessed and reported separately for different groups of people, are conceptually simple.
Greedy Algorithm almost Dominates in Smoothed Contextual Bandits
Online learning algorithms, widely used to power search and content optimization on the web, must balance exploration and exploitation, potentially sacrificing the experience of current users in order to gain information that will lead to better decisions in the future.
No-Regret and Incentive-Compatible Online Learning
First, we want the learning algorithm to be no-regret with respect to the best fixed expert in hindsight.
Weight of Evidence as a Basis for Human-Oriented Explanations
Interpretability is an elusive but highly sought-after characteristic of modern machine learning methods.
Toward Fairness in AI for People with Disabilities: A Research Roadmap
AI technologies have the potential to dramatically impact the lives of people with disabilities (PWD).
Improving fairness in machine learning systems: What do industry practitioners need?
The potential for machine learning (ML) systems to amplify social inequities and unfairness is receiving increasing popular and academic attention.
The Disparate Effects of Strategic Manipulation
When consequential decisions are informed by algorithmic input, individuals may feel compelled to alter their behavior in order to gain a system's approval.
Using Search Queries to Understand Health Information Needs in Africa
The lack of comprehensive, high-quality health data in developing nations creates a roadblock for combating the impacts of disease.
The Externalities of Exploration and How Data Diversity Helps Exploitation
Returning to group-level effects, we show that under the same conditions, negative group externalities essentially vanish under the greedy algorithm.
Datasheets for Datasets
The machine learning community currently has no standardized process for documenting datasets, which can lead to severe consequences in high-stakes domains.
Manipulating and Measuring Model Interpretability
With machine learning models being increasingly used to aid decision making even in high-stakes domains, there has been a growing interest in developing interpretable models.
Tutorial: Making Better Use of the Crowd
Over the last decade, crowdsourcing has been used to harness the power of human computation to solve tasks that are notoriously difficult to solve with computers alone, such as determining whether or not an image contains a tree, rating the relevance of a website, or verifying the phone number of a business.
Oracle-Efficient Online Learning and Auction Design
We consider the design of computationally efficient online learning algorithms in an adversarial setting in which the learner has access to an offline optimization oracle.
The Possibilities and Limitations of Private Prediction Markets
We consider the design of private prediction markets, financial markets designed to elicit predictions about uncertain events without revealing too much information about market participants' actions or beliefs.
Market Making with Decreasing Utility for Information
We study information elicitation in cost-function-based combinatorial prediction markets when the market maker's utility for information decreases over time.
Adaptive Contract Design for Crowdsourcing Markets: Bandit Algorithms for Repeated Principal-Agent Problems
In this paper, we study the requester's problem of dynamically adjusting quality-contingent payments for tasks.
