Search Results for author:
Found 37 papers, 6 papers with code
The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGD
The standard composition-based privacy analysis of DP-SGD effectively assumes that the adversary has access to all intermediate iterates, which is often unrealistic.
Private Geometric Median
Our main contribution is a pair of polynomial-time DP algorithms for the task of private GM with an excess error guarantee that scales with the effective diameter of the datapoints.
Avoiding Pitfalls for Privacy Accounting of Subsampled Mechanisms under Composition
We show that the privacy guarantees may in fact differ significantly between the two sampling schemes.
Efficient and Near-Optimal Noise Generation for Streaming Differential Privacy
Existing algorithms for differentially private continual counting are either inefficient in terms of their space usage or add an excessive amount of noise, inducing suboptimal utility.
Privacy Amplification for Matrix Mechanisms
In this paper, we propose "MMCC", the first algorithm to analyze privacy amplification via sampling for any generic matrix mechanism.
Correlated Noise Provably Beats Independent Noise for Differentially Private Learning
We characterize the asymptotic learning utility for any choice of the correlation function, giving precise analytical bounds for linear regression and as the solution to a convex program for general convex functions.
Differentially Private Medians and Interior Points for Non-Pathological Data
We construct differentially private estimators with low sample complexity that estimate the median of an arbitrary distribution over $\mathbb{R}$ satisfying very mild moment conditions.
Why Is Public Pretraining Necessary for Private Model Training?
To explain this phenomenon, we hypothesize that the non-convex loss landscape of a model training necessitates an optimization algorithm to go through two phases.
Tight Auditing of Differentially Private Machine Learning
Moreover, our auditing scheme requires only two training runs (instead of thousands) to produce tight privacy estimates, by adapting recent advances in tight composition theorems for differential privacy.
A Bias-Accuracy-Privacy Trilemma for Statistical Estimation
Differential privacy (DP) is a rigorous notion of data privacy, used for private statistics.
Composition of Differential Privacy & Privacy Amplification by Subsampling
This chapter is meant to be part of the book "Differential Privacy for Artificial Intelligence Applications."
Algorithms with More Granular Differential Privacy Guarantees
Differential privacy is often applied with a privacy parameter that is larger than the theory suggests is ideal; various informal justifications for tolerating large privacy parameters have been proposed.
Debugging Differential Privacy: A Case Study for Privacy Auditing
Differential Privacy can provide provable privacy guarantees for training data in machine learning.
Public Data-Assisted Mirror Descent for Private Model Training
In this paper, we revisit the problem of using in-distribution public data to improve the privacy/utility trade-offs for differentially private (DP) model training.
A Private and Computationally-Efficient Estimator for Unbounded Gaussians
We give the first polynomial-time, polynomial-sample, differentially private estimator for the mean and covariance of an arbitrary Gaussian distribution $\mathcal{N}(\mu,\Sigma)$ in $\mathbb{R}^d$.
Hyperparameter Tuning with Renyi Differential Privacy
For many differentially private algorithms, such as the prominent noisy stochastic gradient descent (DP-SGD), the analysis needed to bound the privacy leakage of a single training run is well understood.
PAC-Bayes, MAC-Bayes and Conditional Mutual Information: Fast rate bounds that handle general VC classes
We give a novel, unified derivation of conditional PAC-Bayesian and mutual information (MI) generalization bounds.
Leveraging Public Data for Practical Private Query Release
In many statistical problems, incorporating priors can significantly improve performance.
The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation
To ensure privacy, we add on-device noise and use secure aggregation so that only the noisy sum is revealed to the server.
New Oracle-Efficient Algorithms for Private Synthetic Data Release
We present three new algorithms for constructing differentially private synthetic data---a sanitized version of a sensitive dataset that approximately preserves the answers to a large collection of statistical queries.
Evading Curse of Dimensionality in Unconstrained Private GLMs via Private Gradient Descent
We show that for unconstrained convex generalized linear models (GLMs), one can obtain an excess empirical risk of $\tilde O\left(\sqrt{{\texttt{rank}}}/\epsilon n\right)$, where ${\texttt{rank}}$ is the rank of the feature matrix in the GLM problem, $n$ is the number of data samples, and $\epsilon$ is the privacy parameter.
The Discrete Gaussian for Differential Privacy
Specifically, we theoretically and experimentally show that adding discrete Gaussian noise provides essentially the same privacy and accuracy guarantees as the addition of continuous Gaussian noise.
Reasoning About Generalization via Conditional Mutual Information
We provide an information-theoretic framework for studying the generalization properties of machine learning algorithms.
Average-Case Averages: Private Algorithms for Smooth Sensitivity and Mean Estimation
The simplest and most widely applied method for guaranteeing differential privacy is to add instance-independent noise to a statistic of interest that is scaled to its global sensitivity.
Statistics Theory Cryptography and Security Data Structures and Algorithms Statistics Theory
Private Hypothesis Selection
The sample complexity of our basic algorithm is $O\left(\frac{\log m}{\alpha^2} + \frac{\log m}{\alpha \varepsilon}\right)$, representing a minimal cost for privacy when compared to the non-private algorithm.
A Hybrid Approach to Privacy-Preserving Federated Learning
Federated learning facilitates the collaborative training of models without the sharing of raw data.
The Limits of Post-Selection Generalization
While statistics and machine learning offers numerous methods for ensuring generalization, these methods often fail in the presence of adaptivity---the common practice in which the choice of analysis depends on previous interactions with the same dataset.
Calibrating Noise to Variance in Adaptive Data Analysis
We demonstrate that a simple and natural algorithm based on adding noise scaled to the standard deviation of the query provides our notion of stability.
Generalization for Adaptively-chosen Estimators via Stable Median
We present an algorithm that estimates the expectations of $k$ arbitrary adaptively-chosen real-valued estimators using a number of samples that scales as $\sqrt{k}$.
Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds
"Concentrated differential privacy" was recently introduced by Dwork and Rothblum as a relaxation of differential privacy, which permits sharper analyses of many privacy-preserving computations.
Make Up Your Mind: The Price of Online Queries in Differential Privacy
The queries may be chosen adversarially from a larger set Q of allowable queries in one of three ways, which we list in order from easiest to hardest to answer: Offline: The queries are chosen all at once and the differentially private mechanism answers the queries in a single batch.
Algorithmic Stability for Adaptive Data Analysis
Specifically, suppose there is an unknown distribution $\mathbf{P}$ and a set of $n$ independent samples $\mathbf{x}$ is drawn from $\mathbf{P}$.
More General Queries and Less Generalization Error in Adaptive Data Analysis
However, generalization error is typically bounded in a non-adaptive model, where all questions are specified before the dataset is drawn.
Between Pure and Approximate Differential Privacy
The novelty of our bound is that it depends optimally on the parameter $\delta$, which loosely corresponds to the probability that the algorithm fails to be private, and is the first to smoothly interpolate between approximate differential privacy ($\delta > 0$) and pure differential privacy ($\delta = 0$).
Weighted Polynomial Approximations: Limits for Learning and Pseudorandomness
The power of this algorithm relies on the fact that under log-concave distributions, halfspaces can be approximated arbitrarily well by low-degree polynomials.
Interactive Fingerprinting Codes and the Hardness of Preventing False Discovery
We show an essentially tight bound on the number of adaptively chosen statistical queries that a computationally efficient algorithm can answer accurately given $n$ samples from an unknown distribution.
