Search Results for author: John Duchi

Found 25 papers, 6 papers with code

Fine-tuning is Fine in Federated Learning

no code implementations16 Aug 2021 Gary Cheng, Karan Chadha, John Duchi

Our starting point is the formulation of federated learning as a multi-criterion objective, where the goal is to minimize each client's loss using information from all of the clients.

Federated Learning Meta-Learning

Adapting to Function Difficulty and Growth Conditions in Private Optimization

no code implementations5 Aug 2021 Hilal Asi, Daniel Levy, John Duchi

We develop algorithms for private stochastic convex optimization that adapt to the hardness of the specific function we wish to optimize.

Private Adaptive Gradient Methods for Convex Optimization

no code implementations25 Jun 2021 Hilal Asi, John Duchi, Alireza Fallah, Omid Javidbakht, Kunal Talwar

We study adaptive methods for differentially private convex optimization, proposing and analyzing differentially private variants of a Stochastic Gradient Descent (SGD) algorithm with adaptive stepsizes, as well as the AdaGrad algorithm.

On Misspecification in Prediction Problems and Robustness via Improper Learning

no code implementations13 Jan 2021 Annie Marsden, John Duchi, Gregory Valiant

We study probabilistic prediction games when the underlying model is misspecified, investigating the consequences of predicting using an incorrect parametric model.

Neural Bridge Sampling for Evaluating Safety-Critical Autonomous Systems

no code implementations NeurIPS 2020 Aman Sinha, Matthew O'Kelly, Russ Tedrake, John Duchi

Learning-based methodologies increasingly find applications in safety-critical domains like autonomous driving and medical robotics.

Autonomous Driving

Distributionally Robust Losses for Latent Covariate Mixtures

no code implementations28 Jul 2020 John Duchi, Tatsunori Hashimoto, Hongseok Namkoong

While modern large-scale datasets often consist of heterogeneous subpopulations---for example, multiple demographic groups or multiple text corpora---the standard practice of minimizing average loss fails to guarantee uniformly low losses across all subpopulations.

Knowing what you know: valid and validated confidence sets in multiclass and multilabel prediction

no code implementations21 Apr 2020 Maxime Cauchois, Suyash Gupta, John Duchi

We develop conformal prediction methods for constructing valid predictive confidence sets in multiclass and multilabel problems without assumptions on the data generating distribution.

Understanding and Mitigating the Tradeoff Between Robustness and Accuracy

1 code implementation ICML 2020 Aditi Raghunathan, Sang Michael Xie, Fanny Yang, John Duchi, Percy Liang

In this work, we precisely characterize the effect of augmentation on the standard error in linear regression when the optimal linear predictor has zero standard and robust error.

Element Level Differential Privacy: The Right Granularity of Privacy

no code implementations5 Dec 2019 Hilal Asi, John Duchi, Omid Javidbakht

Differential Privacy (DP) provides strong guarantees on the risk of compromising a user's data in statistical learning applications, though these strong protections make learning challenging and may be too stringent for some use cases.

Proximal algorithms for constrained composite optimization, with applications to solving low-rank SDPs

no code implementations1 Mar 2019 Yu Bai, John Duchi, Song Mei

We study a family of (potentially non-convex) constrained optimization problems with convex composite structure.

Protection Against Reconstruction and Its Applications in Private Federated Learning

no code implementations3 Dec 2018 Abhishek Bhowmick, John Duchi, Julien Freudiger, Gaurav Kapoor, Ryan Rogers

In large-scale statistical learning, data collection and model fitting are moving increasingly toward peripheral devices---phones, watches, fitness trackers---away from centralized data collection.

Federated Learning Image Classification

Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation

2 code implementations NeurIPS 2018 Matthew O'Kelly, Aman Sinha, Hongseok Namkoong, John Duchi, Russ Tedrake

While recent developments in autonomous vehicle (AV) technology highlight substantial progress, we lack tools for rigorous and scalable testing.

Autonomous Driving

Learning Models with Uniform Performance via Distributionally Robust Optimization

no code implementations20 Oct 2018 John Duchi, Hongseok Namkoong

A common goal in statistics and machine learning is to learn models that can perform well against distributional shifts, such as latent heterogeneous subpopulations, unknown covariate shifts, or unmodeled temporal effects.

Stochastic Optimization

Generalizing to Unseen Domains via Adversarial Data Augmentation

2 code implementations NeurIPS 2018 Riccardo Volpi, Hongseok Namkoong, Ozan Sener, John Duchi, Vittorio Murino, Silvio Savarese

Only using training data from a single source distribution, we propose an iterative procedure that augments the dataset with examples from a fictitious target domain that is "hard" under the current model.

Data Augmentation Semantic Segmentation

Certifying Some Distributional Robustness with Principled Adversarial Training

no code implementations ICLR 2018 Aman Sinha, Hongseok Namkoong, Riccardo Volpi, John Duchi

Neural networks are vulnerable to adversarial examples and researchers have proposed many heuristic attack and defense mechanisms.

Asymptotic Optimality in Stochastic Optimization

no code implementations16 Dec 2016 John Duchi, Feng Ruan

We study local complexity measures for stochastic convex optimization problems, providing a local minimax theory analogous to that of H\'{a}jek and Le Cam for classical statistical problems.

Stochastic Optimization

Statistics of Robust Optimization: A Generalized Empirical Likelihood Approach

no code implementations11 Oct 2016 John Duchi, Peter Glynn, Hongseok Namkoong

We study statistical inference and distributionally robust solution methods for stochastic optimization problems, focusing on confidence intervals for optimal values and solutions that achieve exact coverage asymptotically.

Stochastic Optimization

Variance-based regularization with convex objectives

1 code implementation NeurIPS 2017 John Duchi, Hongseok Namkoong

We develop an approach to risk minimization and stochastic optimization that provides a convex surrogate for variance, allowing near-optimal and computationally efficient trading between approximation and estimation error.

General Classification Stochastic Optimization

Estimation from Indirect Supervision with Linear Moments

1 code implementation10 Aug 2016 Aditi Raghunathan, Roy Frostig, John Duchi, Percy Liang

In structured prediction problems where we have indirect supervision of the output, maximum marginal likelihood faces two computational obstacles: non-convexity of the objective and intractability of even a single gradient computation.

Structured Prediction

Local Minimax Complexity of Stochastic Convex Optimization

no code implementations NeurIPS 2016 Yuancheng Zhu, Sabyasachi Chatterjee, John Duchi, John Lafferty

The bounds are expressed in terms of a localized and computational analogue of the modulus of continuity that is central to statistical minimax analysis.

Local Privacy and Minimax Bounds: Sharp Rates for Probability Estimation

no code implementations NeurIPS 2013 John Duchi, Martin J. Wainwright, Michael. I. Jordan

We provide a detailed study of the estimation of probability distributions---discrete and continuous---in a stringent setting in which data is kept private even from the statistician.

Survey Sampling

Information-theoretic lower bounds for distributed statistical estimation with communication constraints

no code implementations NeurIPS 2013 Yuchen Zhang, John Duchi, Michael. I. Jordan, Martin J. Wainwright

We establish minimax risk lower bounds for distributed statistical estimation given a budget $B$ of the total number of bits that may be communicated.

General Classification

Estimation, Optimization, and Parallelism when Data is Sparse

no code implementations NeurIPS 2013 John Duchi, Michael. I. Jordan, Brendan Mcmahan

We study stochastic optimization problems when the \emph{data} is sparse, which is in a sense dual to the current understanding of high-dimensional statistical learning and optimization.

Stochastic Optimization

Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling

no code implementations12 May 2010 John Duchi, Alekh Agarwal, Martin Wainwright

The goal of decentralized optimization over a network is to optimize a global objective formed by a sum of local (possibly nonsmooth) convex functions using only local computation and communication.

Distributed Optimization

Cannot find the paper you are looking for? You can Submit a new open access paper.