Search Results for author: Aaron Roth

Found 62 papers, 19 papers with code

Individually Fair Learning with One-Sided Feedback

no code implementations9 Jun 2022 Yahav Bechavod, Aaron Roth

We consider an online learning problem with one-sided feedback, in which the learner is able to observe the true label only for positively predicted instances.

Fairness online learning

Practical Adversarial Multivalid Conformal Prediction

1 code implementation2 Jun 2022 Osbert Bastani, Varun Gupta, Christopher Jung, Georgy Noarov, Ramya Ramalingam, Aaron Roth

It is computationally lightweight -- comparable to split conformal prediction -- but does not require having a held-out validation set, and so all data can be used for training models from which to derive a conformal score.

Mixed Differential Privacy in Computer Vision

no code implementations CVPR 2022 Aditya Golatkar, Alessandro Achille, Yu-Xiang Wang, Aaron Roth, Michael Kearns, Stefano Soatto

AdaMix incorporates few-shot training, or cross-modal zero-shot learning, on public data prior to private fine-tuning, to improve the trade-off.

Computer Vision Zero-Shot Learning

An Algorithmic Framework for Bias Bounties

no code implementations25 Jan 2022 Ira Globus-Harris, Michael Kearns, Aaron Roth

We propose and analyze an algorithmic framework for "bias bounties": events in which external participants are invited to propose improvements to a trained model, akin to bug bounty events in software and security.

Fairness

Multiaccurate Proxies for Downstream Fairness

no code implementations9 Jul 2021 Emily Diana, Wesley Gill, Michael Kearns, Krishnaram Kenthapadi, Aaron Roth, Saeed Sharifi-Malvajerdi

The goal of the proxy is to allow a general "downstream" learner -- with minimal assumptions on their prediction task -- to be able to use the proxy to train a model that is fair with respect to the true sensitive features.

Fairness Generalization Bounds

Adaptive Machine Unlearning

1 code implementation NeurIPS 2021 Varun Gupta, Christopher Jung, Seth Neel, Aaron Roth, Saeed Sharifi-Malvajerdi, Chris Waites

In this paper, we give a general reduction from deletion guarantees against adaptive sequences to deletion guarantees against non-adaptive sequences, using differential privacy and its connection to max information.

Rejoinder: Gaussian Differential Privacy

no code implementations5 Apr 2021 Jinshuo Dong, Aaron Roth, Weijie J. Su

In this rejoinder, we aim to address two broad issues that cover most comments made in the discussion.

Privacy Preserving

Differentially Private Query Release Through Adaptive Projection

1 code implementation11 Mar 2021 Sergul Aydore, William Brown, Michael Kearns, Krishnaram Kenthapadi, Luca Melis, Aaron Roth, Ankit Siva

We propose, implement, and evaluate a new algorithm for releasing answers to very large numbers of statistical queries like $k$-way marginals, subject to differential privacy.

Lexicographically Fair Learning: Algorithms and Generalization

no code implementations16 Feb 2021 Emily Diana, Wesley Gill, Ira Globus-Harris, Michael Kearns, Aaron Roth, Saeed Sharifi-Malvajerdi

We extend the notion of minimax fairness in supervised learning problems to its natural conclusion: lexicographic minimax fairness (or lexifairness for short).

Fairness Generalization Bounds

Online Multivalid Learning: Means, Moments, and Prediction Intervals

no code implementations5 Jan 2021 Varun Gupta, Christopher Jung, Georgy Noarov, Mallesh M. Pai, Aaron Roth

We present a general, efficient technique for providing contextual predictions that are "multivalid" in various senses, against an online sequence of adversarially chosen examples $(x, y)$.

Prediction Intervals

Minimax Group Fairness: Algorithms and Experiments

1 code implementation5 Nov 2020 Emily Diana, Wesley Gill, Michael Kearns, Krishnaram Kenthapadi, Aaron Roth

We consider a recently introduced framework in which fairness is measured by worst-case outcomes across groups, rather than by the more standard differences between group outcomes.

Fairness

Moment Multicalibration for Uncertainty Estimation

no code implementations18 Aug 2020 Christopher Jung, Changhwa Lee, Mallesh M. Pai, Aaron Roth, Rakesh Vohra

We show how to achieve the notion of "multicalibration" from H\'ebert-Johnson et al. [2018] not just for means, but also for variances and other higher moments.

Prediction Intervals

Fair Prediction with Endogenous Behavior

no code implementations18 Feb 2020 Christopher Jung, Sampath Kannan, Changhwa Lee, Mallesh M. Pai, Aaron Roth, Rakesh Vohra

There is increasing regulatory interest in whether machine learning algorithms deployed in consequential domains (e. g. in criminal justice) treat different demographic groups "fairly."

Fairness

Pipeline Interventions

no code implementations16 Feb 2020 Eshwar Ram Arunachaleswaran, Sampath Kannan, Aaron Roth, Juba Ziani

We consider two objectives: social welfare maximization, and a fairness-motivated maximin objective that seeks to maximize the value to the population (starting node) with the \emph{least} expected value.

Fairness

Optimal, Truthful, and Private Securities Lending

no code implementations12 Dec 2019 Emily Diana, Michael Kearns, Seth Neel, Aaron Roth

We consider a fundamental dynamic allocation problem motivated by the problem of $\textit{securities lending}$ in financial markets, the mechanism underlying the short selling of stocks.

A New Analysis of Differential Privacy's Generalization Guarantees

no code implementations9 Sep 2019 Christopher Jung, Katrina Ligett, Seth Neel, Aaron Roth, Saeed Sharifi-Malvajerdi, Moshe Shenfeld

This second claim follows from a thought experiment in which we imagine that the dataset is resampled from the posterior distribution after the mechanism has committed to its answers.

Oracle Efficient Private Non-Convex Optimization

1 code implementation ICML 2020 Seth Neel, Aaron Roth, Giuseppe Vietri, Zhiwei Steven Wu

We find that for the problem of learning linear classifiers, directly optimizing for 0/1 loss using our approach can out-perform the more standard approach of privately optimizing a convex-surrogate loss function on the Adult dataset.

Exponential Separations in Local Differential Privacy

no code implementations1 Jul 2019 Matthew Joseph, Jieming Mao, Aaron Roth

We prove a general connection between the communication complexity of two-player games and the sample complexity of their multi-player locally private analogues.

Guaranteed Validity for Empirical Approaches to Adaptive Data Analysis

1 code implementation21 Jun 2019 Ryan Rogers, Aaron Roth, Adam Smith, Nathan Srebro, Om Thakkar, Blake Woodworth

We design a general framework for answering adaptive statistical queries that focuses on providing explicit confidence intervals along with point estimates.

Fuzzi: A Three-Level Logic for Differential Privacy

no code implementations29 May 2019 Hengchu Zhang, Edo Roth, Andreas Haeberlen, Benjamin C. Pierce, Aaron Roth

Curators of sensitive datasets sometimes need to know whether queries against the data are differentially private [Dwork et al. 2006].

Programming Languages Logic in Computer Science

Average Individual Fairness: Algorithms, Generalization and Experiments

1 code implementation NeurIPS 2019 Michael Kearns, Aaron Roth, Saeed Sharifi-Malvajerdi

Given a sample of individuals and classification problems, we design an oracle-efficient algorithm (i. e. one that is given access to any standard, fairness-free learning heuristic) for the fair empirical risk minimization task.

Classification Fairness +1

An Algorithmic Framework for Fairness Elicitation

1 code implementation25 May 2019 Christopher Jung, Michael Kearns, Seth Neel, Aaron Roth, Logan Stapleton, Zhiwei Steven Wu

We consider settings in which the right notion of fairness is not captured by simple mathematical definitions (such as equality of error rates across groups), but might be more complex and nuanced and thus require elicitation from individual or collective stakeholders.

Fairness Generalization Bounds

Gaussian Differential Privacy

3 code implementations7 May 2019 Jinshuo Dong, Aaron Roth, Weijie J. Su

More precisely, the privacy guarantees of \emph{any} hypothesis testing based definition of privacy (including original DP) converges to GDP in the limit under composition.

Two-sample testing

The Role of Interactivity in Local Differential Privacy

no code implementations7 Apr 2019 Matthew Joseph, Jieming Mao, Seth Neel, Aaron Roth

Next, we show that our reduction is tight by exhibiting a family of problems such that for any $k$, there is a fully interactive $k$-compositional protocol which solves the problem, while no sequentially interactive protocol can solve the problem without at least an $\tilde \Omega(k)$ factor more examples.

Two-sample testing

Equal Opportunity in Online Classification with Partial Feedback

1 code implementation NeurIPS 2019 Yahav Bechavod, Katrina Ligett, Aaron Roth, Bo Waggoner, Zhiwei Steven Wu

We study an online classification problem with partial feedback in which individuals arrive one at a time from a fixed but unknown distribution, and must be classified as positive or negative.

Classification Decision Making Under Uncertainty +3

Differentially Private Fair Learning

no code implementations6 Dec 2018 Matthew Jagielski, Michael Kearns, Jieming Mao, Alina Oprea, Aaron Roth, Saeed Sharifi-Malvajerdi, Jonathan Ullman

This algorithm is appealingly simple, but must be able to use protected group membership explicitly at test time, which can be viewed as a form of 'disparate treatment'.

Fairness

How to Use Heuristics for Differential Privacy

no code implementations19 Nov 2018 Seth Neel, Aaron Roth, Zhiwei Steven Wu

We show that there is an efficient algorithm for privately constructing synthetic data for any such class, given a non-private learning oracle.

PAC learning

The Frontiers of Fairness in Machine Learning

no code implementations20 Oct 2018 Alexandra Chouldechova, Aaron Roth

The last few years have seen an explosion of academic and popular interest in algorithmic fairness.

Fairness

Fair Algorithms for Learning in Allocation Problems

no code implementations30 Aug 2018 Hadi Elzayn, Shahin Jabbari, Christopher Jung, Michael Kearns, Seth Neel, Aaron Roth, Zachary Schutzman

We formalize this fairness notion for allocation problems and investigate its algorithmic consequences.

Fairness

Downstream Effects of Affirmative Action

no code implementations27 Aug 2018 Sampath Kannan, Aaron Roth, Juba Ziani

We show that both goals can be achieved when the college does not report grades.

Fairness

An Empirical Study of Rich Subgroup Fairness for Machine Learning

5 code implementations24 Aug 2018 Michael Kearns, Seth Neel, Aaron Roth, Zhiwei Steven Wu

In this paper, we undertake an extensive empirical evaluation of the algorithm of Kearns et al. On four real datasets for which fairness is a concern, we investigate the basic convergence of the algorithm when instantiated with fast heuristics in place of learning oracles, measure the tradeoffs between fairness and accuracy, and compare this approach with the recent algorithm of Agarwal et al. [2018], which implements weaker and more traditional marginal fairness constraints defined by individual protected attributes.

Fairness

Online Learning with an Unknown Fairness Metric

no code implementations NeurIPS 2018 Stephen Gillen, Christopher Jung, Michael Kearns, Aaron Roth

We consider the problem of online learning in the linear contextual bandits setting, but in which there are also strong individual fairness constraints governed by an unknown similarity metric.

Fairness Multi-Armed Bandits +1

Local Differential Privacy for Evolving Data

no code implementations NeurIPS 2018 Matthew Joseph, Aaron Roth, Jonathan Ullman, Bo Waggoner

Moreover, existing techniques to mitigate this effect do not apply in the "local model" of differential privacy that these systems use.

Accuracy First: Selecting a Differential Privacy Level for Accuracy Constrained ERM

no code implementations NeurIPS 2017 Katrina Ligett, Seth Neel, Aaron Roth, Bo Waggoner, Steven Z. Wu

Traditional approaches to differential privacy assume a fixed privacy requirement ε for a computation, and attempt to maximize the accuracy of the computation subject to the privacy constraint.

Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness

5 code implementations ICML 2018 Michael Kearns, Seth Neel, Aaron Roth, Zhiwei Steven Wu

We prove that the computational problem of auditing subgroup fairness for both equality of false positive rates and statistical parity is equivalent to the problem of weak agnostic learning, which means it is computationally hard in the worst case, even for simple structured subclasses.

Fairness

Strategic Classification from Revealed Preferences

no code implementations22 Oct 2017 Jinshuo Dong, Aaron Roth, Zachary Schutzman, Bo Waggoner, Zhiwei Steven Wu

We study an online linear classification problem, in which the data is generated by strategic agents who manipulate their features in an effort to change the classification outcome.

Classification General Classification

Meritocratic Fairness for Cross-Population Selection

no code implementations ICML 2017 Michael Kearns, Aaron Roth, Zhiwei Steven Wu

We consider the problem of selecting a strong pool of individuals from several populations with incomparable skills (e. g. soccer players, mathematicians, and singers) in a fair manner.

Fairness

A Convex Framework for Fair Regression

1 code implementation7 Jun 2017 Richard Berk, Hoda Heidari, Shahin Jabbari, Matthew Joseph, Michael Kearns, Jamie Morgenstern, Seth Neel, Aaron Roth

We introduce a flexible family of fairness regularizers for (linear and logistic) regression problems.

Fairness

Accuracy First: Selecting a Differential Privacy Level for Accuracy-Constrained ERM

1 code implementation30 May 2017 Katrina Ligett, Seth Neel, Aaron Roth, Bo Waggoner, Z. Steven Wu

Traditional approaches to differential privacy assume a fixed privacy requirement $\epsilon$ for a computation, and attempt to maximize the accuracy of the computation subject to the privacy constraint.

Fairness in Criminal Justice Risk Assessments: The State of the Art

no code implementations27 Mar 2017 Richard Berk, Hoda Heidari, Shahin Jabbari, Michael Kearns, Aaron Roth

Methods: We draw on the existing literatures in criminology, computer science and statistics to provide an integrated examination of fairness and accuracy in criminal justice risk assessments.

Fairness

Fairness in Reinforcement Learning

no code implementations ICML 2017 Shahin Jabbari, Matthew Joseph, Michael Kearns, Jamie Morgenstern, Aaron Roth

We initiate the study of fairness in reinforcement learning, where the actions of a learning algorithm may affect its environment and future rewards.

Fairness reinforcement-learning

Multidimensional Dynamic Pricing for Welfare Maximization

no code implementations19 Jul 2016 Aaron Roth, Aleksandrs Slivkins, Jonathan Ullman, Zhiwei Steven Wu

We are able to apply this technique to the setting of unit demand buyers despite the fact that in that setting the goods are not divisible, and the natural fractional relaxation of a unit demand valuation is not strongly concave.

Fairness in Learning: Classic and Contextual Bandits

no code implementations NeurIPS 2016 Matthew Joseph, Michael Kearns, Jamie Morgenstern, Aaron Roth

This tight connection allows us to provide a provably fair algorithm for the linear contextual bandit problem with a polynomial dependence on the dimension, and to show (for a different class of functions) a worst-case exponential gap in regret between fair and non-fair learning algorithms

Fairness Multi-Armed Bandits

Max-Information, Differential Privacy, and Post-Selection Hypothesis Testing

no code implementations13 Apr 2016 Ryan Rogers, Aaron Roth, Adam Smith, Om Thakkar

In this paper, we initiate a principled study of how the generalization properties of approximate differential privacy can be used to perform adaptive hypothesis testing, while giving statistically valid $p$-value corrections.

Two-sample testing

Adaptive Learning with Robust Generalization Guarantees

no code implementations24 Feb 2016 Rachel Cummings, Katrina Ligett, Kobbi Nissim, Aaron Roth, Zhiwei Steven Wu

We also show that perfect generalization is a strictly stronger guarantee than differential privacy, but that, nevertheless, many learning tasks can be carried out subject to the guarantees of perfect generalization.

Do Prices Coordinate Markets?

no code implementations3 Nov 2015 Justin Hsu, Jamie Morgenstern, Ryan Rogers, Aaron Roth, Rakesh Vohra

Second, we provide learning-theoretic results that show that such prices are robust to changing the buyers in the market, so long as all buyers are sampled from the same (unknown) distribution.

Learning from Rational Behavior: Predicting Solutions to Unknown Linear Programs

no code implementations NeurIPS 2016 Shahin Jabbari, Ryan Rogers, Aaron Roth, Zhiwei Steven Wu

This models the problem of predicting the behavior of a rational agent whose goals are known, but whose resources are unknown.

Watch and Learn: Optimizing from Revealed Preferences Feedback

no code implementations4 Apr 2015 Aaron Roth, Jonathan Ullman, Zhiwei Steven Wu

In this paper we present an approach to solving for the leader's optimal strategy in certain Stackelberg games where the follower's utility function (and thus the subsequent best response of the follower) is unknown.

Computer-aided verification in mechanism design

1 code implementation13 Feb 2015 Gilles Barthe, Marco Gaboardi, Emilio Jesús Gallego Arias, Justin Hsu, Aaron Roth, Pierre-Yves Strub

To address both concerns, we explore techniques from computer-aided verification to construct formal proofs of incentive properties.

Computer Science and Game Theory Logic in Computer Science

Preserving Statistical Validity in Adaptive Data Analysis

no code implementations10 Nov 2014 Cynthia Dwork, Vitaly Feldman, Moritz Hardt, Toniann Pitassi, Omer Reingold, Aaron Roth

We show that, surprisingly, there is a way to estimate an exponential in $n$ number of expectations accurately even if the functions are chosen adaptively.

Two-sample testing

Higher-Order Approximate Relational Refinement Types for Mechanism Design and Differential Privacy

1 code implementation25 Jul 2014 Gilles Barthe, Marco Gaboardi, Emilio Jesús Gallego Arias, Justin Hsu, Aaron Roth, Pierre-Yves Strub

Unlike typical programmatic properties, it is not sufficient for algorithms to merely satisfy the property---incentive properties are only useful if the strategic agents also believe this fact.

Programming Languages Computer Science and Game Theory

Privately Solving Linear Programs

no code implementations15 Feb 2014 Justin Hsu, Aaron Roth, Tim Roughgarden, Jonathan Ullman

In this paper, we initiate the systematic study of solving linear programs under differential privacy.

Dual Query: Practical Private Query Release for High Dimensional Data

no code implementations6 Feb 2014 Marco Gaboardi, Emilio Jesús Gallego Arias, Justin Hsu, Aaron Roth, Zhiwei Steven Wu

We present a practical, differentially private algorithm for answering a large number of queries on high dimensional datasets.

Differentially Private Combinatorial Optimization

no code implementations26 Mar 2009 Anupam Gupta, Katrina Ligett, Frank McSherry, Aaron Roth, Kunal Talwar

Is it even possible to design good algorithms for this problem that preserve the privacy of the clients?

Data Structures and Algorithms Cryptography and Security Computer Science and Game Theory

Cannot find the paper you are looking for? You can Submit a new open access paper.