Search Results for author:
Found 8 papers, 0 papers with code
Tight Group-Level DP Guarantees for DP-SGD with Sampling via Mixture of Gaussians Mechanisms
We give a procedure for computing group-level $(\epsilon, \delta)$-DP guarantees for DP-SGD, when using Poisson sampling or fixed batch size sampling.
Privacy Amplification for Matrix Mechanisms
In this paper, we propose "MMCC", the first algorithm to analyze privacy amplification via sampling for any generic matrix mechanism.
Correlated Noise Provably Beats Independent Noise for Differentially Private Learning
We characterize the asymptotic learning utility for any choice of the correlation function, giving precise analytical bounds for linear regression and as the solution to a convex program for general convex functions.
Why Is Public Pretraining Necessary for Private Model Training?
To explain this phenomenon, we hypothesize that the non-convex loss landscape of a model training necessitates an optimization algorithm to go through two phases.
Recycling Scraps: Improving Private Learning by Leveraging Intermediate Checkpoints
Empirically, we show that the last few checkpoints can provide a reasonable lower bound for the variance of a converged DP model.
Differentially Private Sampling from Rashomon Sets, and the Universality of Langevin Diffusion for Convex Optimization
In this paper we provide an algorithmic framework based on Langevin diffusion (LD) and its corresponding discretizations that allow us to simultaneously obtain: i) An algorithm for sampling from the exponential mechanism, whose privacy analysis does not depend on convexity and which can be stopped at anytime without compromising privacy, and ii) tight uniform stability guarantees for the exponential mechanism.
Public Data-Assisted Mirror Descent for Private Model Training
In this paper, we revisit the problem of using in-distribution public data to improve the privacy/utility trade-offs for differentially private (DP) model training.
Faster Differentially Private Samplers via Rényi Divergence Analysis of Discretized Langevin MCMC
Various differentially private algorithms instantiate the exponential mechanism, and require sampling from the distribution $\exp(-f)$ for a suitable function $f$.
