Search Results for author:
Found 34 papers, 0 papers with code
Differentially Private Optimization with Sparse Gradients
Motivated by applications of large embedding models, we study differentially private (DP) optimization problems under sparsity of individual gradients.
How Private is DP-SGD?
We demonstrate a substantial gap between the privacy guarantees of the Adaptive Batch Linear Queries (ABLQ) mechanism under different types of batch sampling: (i) Shuffling, and (ii) Poisson subsampling; the typical analysis of Differentially Private Stochastic Gradient Descent (DP-SGD) follows by interpreting it as a post-processing of ABLQ.
Training Differentially Private Ad Prediction Models with Semi-Sensitive Features
Motivated by problems arising in digital advertising, we introduce the task of training differentially private (DP) machine learning models with semi-sensitive features.
Optimal Unbiased Randomizers for Regression with Label Differential Privacy
We propose a new family of label randomizers for training regression models under the constraint of label differential privacy (DP).
A Note on Hardness of Computing Recursive Teaching Dimension
In this short note, we show that the problem of computing the recursive teaching dimension (RTD) for a concept class (given explicitly as input) requires $n^{\Omega(\log n)}$-time, assuming the exponential time hypothesis (ETH).
Ticketed Learning-Unlearning Schemes
Subsequently, given any subset of examples that wish to be unlearnt, the goal is to learn, without the knowledge of the original training dataset, a good predictor that is identical to the predictor that would have been produced when learning from scratch on the surviving examples.
On User-Level Private Convex Optimization
We introduce a new mechanism for stochastic convex optimization (SCO) with user-level differential privacy guarantees.
Regression with Label Differential Privacy
We study the task of training regression models with the guarantee of label differential privacy (DP).
Private Ad Modeling with DP-SGD
A well-known algorithm in privacy-preserving ML is differentially private stochastic gradient descent (DP-SGD).
Improved Inapproximability of VC Dimension and Littlestone's Dimension via (Unbalanced) Biclique
We study the complexity of computing (and approximating) VC Dimension and Littlestone's Dimension when we are given the concept class explicitly.
Private Isotonic Regression
For the most general problem of isotonic regression over a partially ordered set (poset) $\mathcal{X}$ and for any Lipschitz loss function, we obtain a pure-DP algorithm that, given $n$ input points, has an expected excess empirical risk of roughly $\mathrm{width}(\mathcal{X}) \cdot \log|\mathcal{X}| / n$, where $\mathrm{width}(\mathcal{X})$ is the width of the poset.
Anonymized Histograms in Intermediate Privacy Models
We study the problem of privately computing the anonymized histogram (a. k. a.
Algorithms with More Granular Differential Privacy Guarantees
Differential privacy is often applied with a privacy parameter that is larger than the theory suggests is ideal; various informal justifications for tolerating large privacy parameters have been proposed.
Cryptographic Hardness of Learning Halfspaces with Massart Noise
We study the complexity of PAC learning halfspaces in the presence of Massart noise.
Connect the Dots: Tighter Discrete Approximations of Privacy Loss Distributions
The privacy loss distribution (PLD) provides a tight characterization of the privacy loss of a mechanism in the context of differential privacy (DP).
Faster Privacy Accounting via Evolving Discretization
We introduce a new algorithm for numerical composition of privacy random variables, useful for computing the accurate differential privacy parameters for composition of mechanisms.
Private Robust Estimation by Stabilizing Convex Relaxations
Prior works obtained private robust algorithms for mean estimation of subgaussian distributions with bounded covariance.
User-Level Differentially Private Learning via Correlated Sampling
Most works in learning with differential privacy (DP) have focused on the setting where each user has a single sample.
User-Level Private Learning via Correlated Sampling
Most works in learning with differential privacy (DP) have focused on the setting where each user has a single sample.
Large-Scale Differentially Private BERT
In this work, we study the large-scale pretraining of BERT-Large with differentially private SGD (DP-SGD).
Contextual Recommendations and Low-Regret Cutting-Plane Algorithms
We design algorithms for this problem which achieve regret $O(d\log T)$ and $\exp(O(d \log d))$.
Locally Private k-Means in One Round
We provide an approximation algorithm for k-means clustering in the one-round (aka non-interactive) local model of differential privacy (DP).
Deep Learning with Label Differential Privacy
The Randomized Response (RR) algorithm is a classical technique to improve robustness in survey aggregation, and has been widely adopted in applications with differential privacy guarantees.
On Avoiding the Union Bound When Answering Multiple Differentially Private Queries
On the other hand, the algorithm of Dagan and Kur has a remarkable advantage that the $\ell_{\infty}$ error bound of $O(\frac{1}{\epsilon}\sqrt{k \log \frac{1}{\delta}})$ holds not only in expectation but always (i. e., with probability one) while we can only get a high probability (or expected) guarantee on the error.
Sample-efficient proper PAC learning with approximate differential privacy
In this paper we prove that the sample complexity of properly learning a class of Littlestone dimension $d$ with approximate differential privacy is $\tilde O(d^6)$, ignoring privacy and accuracy parameters.
Robust and Private Learning of Halfspaces
In this work, we study the trade-off between differential privacy and adversarial robustness under L2-perturbations in the context of learning halfspaces.
Tight Hardness Results for Training Depth-2 ReLU Networks
We are also able to obtain lower bounds on the running time in terms of the desired additive error $\epsilon$.
On Distributed Differential Privacy and Counting Distinct Elements
We study the setup where each of $n$ users holds an element from a discrete set, and the goal is to count the number of distinct elements across all users, under the constraint of $(\epsilon, \delta)$-differentially privacy: - In the non-interactive local setting, we prove that the additive error of any protocol is $\Omega(n)$ for any constant $\epsilon$ and for any $\delta$ inverse polynomial in $n$.
Differentially Private Clustering: Tight Approximation Ratios
For several basic clustering problems, including Euclidean DensestBall, 1-Cluster, k-means, and k-median, we give efficient differentially private algorithms that achieve essentially the same approximation ratios as those that can be obtained by any non-private algorithm, while incurring only small additive errors.
The Complexity of Adversarially Robust Proper Learning of Halfspaces with Agnostic Noise
We study the computational complexity of adversarially robust proper learning of halfspaces in the distribution-independent agnostic PAC model, with a focus on $L_p$ perturbations.
Near-tight closure bounds for Littlestone and threshold dimensions
We study closure properties for the Littlestone and threshold dimensions of binary hypothesis classes.
Private Aggregation from Fewer Anonymous Messages
Using a reduction of Balle et al. (2019), our improved analysis of the protocol of Ishai et al. yields, in the same model, an $\left(\varepsilon, \delta\right)$-differentially private protocol for aggregation that, for any constant $\varepsilon > 0$ and any $\delta = \frac{1}{\mathrm{poly}(n)}$, incurs only a constant error and requires only a constant number of messages per party.
Cryptography and Security Data Structures and Algorithms
Nearly Tight Bounds for Robust Proper Learning of Halfspaces with a Margin
We study the problem of {\em properly} learning large margin halfspaces in the agnostic PAC model.
Parameterized Approximation Algorithms for Bidirected Steiner Network Problems
We give a tight inapproximability result by showing that for $k$ no parameterized $(2-\varepsilon)$-approximation algorithm exists under Gap-ETH.
Data Structures and Algorithms Computational Complexity
