Search Results for author:
Found 51 papers, 13 papers with code
Differentially Private Combinatorial Optimization
Is it even possible to design good algorithms for this problem that preserve the privacy of the clients?
Data Structures and Algorithms Cryptography and Security Computer Science and Game Theory
Analyze Gauss: Optimal Bounds for Privacy-Preserving Principal Component Analysis
We show that the well-known, but misnamed, randomized response algorithm, with properly tuned parameters, provides a nearly optimal additive quality gap compared to the best possible singular subspace of A.
Private Empirical Risk Minimization Beyond the Worst Case: The Effect of the Constraint Set Geometry
In addition, we show that when the loss function is Lipschitz with respect to the $\ell_1$ norm and $\mathcal{C}$ is $\ell_1$-bounded, a differentially private version of the Frank-Wolfe algorithm gives error bounds of the form $\tilde{O}(n^{-2/3})$.
Nearly Optimal Private LASSO
In addition, we show that this error bound is nearly optimal amongst all differentially private algorithms.
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
TensorFlow is an interface for expressing machine learning algorithms, and an implementation for executing such algorithms.
Sketching and Neural Networks
In stark contrast, our approach of using improper learning, using a larger hypothesis class allows the sketch size to have a logarithmic dependence on the degree.
Deep Learning with Differential Privacy
Machine learning techniques based on neural networks are achieving remarkable results in a wide variety of domains.
Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data
The approach combines, in a black-box fashion, multiple models trained with disjoint datasets, such as records from different subsets of users.
On the Protection of Private Information in Machine Learning Systems: Two Recent Approaches
The recent, remarkable growth of machine learning has led to intense interest in the privacy of the data on which machine learning relies, and to new techniques for preserving privacy.
Learning Differentially Private Recurrent Language Models
We demonstrate that it is possible to train large recurrent language models with user-level differential privacy guarantees with only a negligible cost in predictive accuracy.
Learning Representations for Faster Similarity Search
Based on our experiments, we propose a number of training modifications that lead to significantly better datasets for nearest neighbor algorithms.
Online learning over a finite action set with limited switching
Using the above result and several reductions, we unify previous work and completely characterize the complexity of this switching budget setting up to small polylogarithmic factors: for both PFE and MAB, for all switching budgets $S \leq T$, and for both expectation and h. p.
Adversarially Robust Generalization Requires More Data
We postulate that the difficulty of training robust classifiers stems, at least partially, from this inherently larger sample complexity.
Online Linear Quadratic Control
We study the problem of controlling linear time-invariant systems with known noisy dynamics and adversarially chosen quadratic losses.
Privacy Amplification by Iteration
In addition, we demonstrate that we can achieve guarantees similar to those obtainable using the privacy-amplification-by-sampling technique in several natural settings where that technique cannot be applied.
Private Selection from Private Candidates
In this work, we consider the selection problem under a much weaker stability assumption on the candidates, namely that the score functions are differentially private.
Amplification by Shuffling: From Local to Central Differential Privacy via Anonymity
We study the collection of such statistics in the local differential privacy (LDP) model, and describe an algorithm whose privacy cost is polylogarithmic in the number of changes to a user's value.
Better Algorithms for Stochastic Bandits with Adversarial Corruptions
We study the stochastic multi-armed bandits problem in the presence of adversarial corruption.
Semi-Cyclic Stochastic Gradient Descent
We consider convex SGD updates with a block-cyclic structure, i. e. where each cycle consists of a small number of blocks, each with many samples from a possibly different, block-specific, distribution.
Private Stochastic Convex Optimization with Optimal Rates
A long line of existing work on private convex optimization focuses on the empirical loss and derives asymptotically tight bounds on the excess empirical loss.
Rényi Differential Privacy of the Sampled Gaussian Mechanism
The Sampled Gaussian Mechanism (SGM)---a composition of subsampling and the additive Gaussian noise---has been successfully used in a number of machine learning applications.
Computational Separations between Sampling and Optimization
Two commonly arising computational tasks in Bayesian learning are Optimization (Maximum A Posteriori estimation) and Sampling (from the posterior distribution).
Characterizing Structural Regularities of Labeled Data in Overparameterized Models
We obtain empirical estimates of this score for individual instances in multiple data sets, and we show that the score identifies out-of-distribution and mislabeled examples at one end of the continuum and strongly regular examples at the other end.
Private Stochastic Convex Optimization: Optimal Rates in Linear Time
We also give a linear-time algorithm achieving the optimal bound on the excess loss for the strongly convex case, as well as a faster algorithm for the non-smooth case.
Stability of Stochastic Gradient Descent on Nonsmooth Convex Losses
Our work is the first to address uniform stability of SGD on {\em nonsmooth} convex losses.
Stochastic Optimization with Laggard Data Pipelines
State-of-the-art optimization is steadily shifting towards massively parallel pipelines with extremely large batch sizes.
Faster Differentially Private Samplers via Rényi Divergence Analysis of Discretized Langevin MCMC
Various differentially private algorithms instantiate the exponential mechanism, and require sampling from the distribution $\exp(-f)$ for a suitable function $f$.
On the Error Resistance of Hinge-Loss Minimization
Commonly used classification algorithms in machine learning, such as support vector machines, minimize a convex surrogate loss on training examples.
On the Error Resistance of Hinge Loss Minimization
Commonly used classification algorithms in machine learning, such as support vector machines, minimize a convex surrogate loss on training examples.
When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning?
Our problems are simple and fairly natural variants of the next-symbol prediction and the cluster labeling tasks.
Hiding Among the Clones: A Simple and Nearly Optimal Analysis of Privacy Amplification by Shuffling
As a direct corollary of our analysis we derive a simple and nearly optimal algorithm for frequency estimation in the shuffle model of privacy.
Lossless Compression of Efficient Private Local Randomizers
Here we demonstrate a general approach that, under standard cryptographic assumptions, compresses every efficient LDP algorithm with negligible loss in privacy and utility guarantees.
Private Stochastic Convex Optimization: Optimal Rates in $\ell_1$ Geometry
Stochastic convex optimization over an $\ell_1$-bounded domain is ubiquitous in machine learning applications such as LASSO but remains poorly understood when learning with differential privacy.
Private Adaptive Gradient Methods for Convex Optimization
We study adaptive methods for differentially private convex optimization, proposing and analyzing differentially private variants of a Stochastic Gradient Descent (SGD) algorithm with adaptive stepsizes, as well as the AdaGrad algorithm.
Differential Secrecy for Distributed Data and Applications to Robust Differentially Secure Vector Summation
In private federated learning applications, these vectors are held by client devices, leading to a distributed summation problem.
Private Frequency Estimation via Projective Geometry
In many parameter settings used in practice this is a significant improvement over the $ O(n+k^2)$ computation cost that is achieved by the recent PI-RAPPOR algorithm (Feldman and Talwar; 2021).
Optimal Algorithms for Mean Estimation under Local Differential Privacy
We show that PrivUnit (Bhowmick et al. 2018) with optimized parameters achieves the optimal variance among a large family of locally private randomizers.
Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Loss
A central issue in machine learning is how to train models on sensitive user data.
FLAIR: Federated Learning Annotated Image Repository
We believe FLAIR can serve as a challenging benchmark for advancing the state-of-the art in federated learning.
Stronger Privacy Amplification by Shuffling for Rényi and Approximate Differential Privacy
Our second contribution is a new analysis of privacy amplification by shuffling.
Resolving the Mixing Time of the Langevin Algorithm to its Stationary Distribution for Log-Concave Sampling
In this way, we disentangle the study of the mixing and bias of the Langevin Algorithm.
Private Online Prediction from Experts: Separations and Faster Rates
Our lower bounds also show a separation between pure and approximate differential privacy for adaptive adversaries where the latter is necessary to achieve the non-private $O(\sqrt{T})$ regret.
Subspace Recovery from Heterogeneous Data with Non-isotropic Noise
Our goal is to recover the linear subspace shared by $\mu_1,\ldots,\mu_n$ using the data points from all users, where every data point from user $i$ is formed by adding an independent mean-zero noise vector to $\mu_i$.
Concentration of the Langevin Algorithm's Stationary Distribution
This discretization leads the Langevin Algorithm to have a stationary distribution $\pi_{\eta}$ which differs from the stationary distribution $\pi$ of the Langevin Diffusion, and it is an important challenge to understand whether the well-known properties of $\pi$ extend to $\pi_{\eta}$.
Near-Optimal Algorithms for Private Online Optimization in the Realizable Regime
We also develop an adaptive algorithm for the small-loss setting with regret $O(L^\star\log d + \varepsilon^{-1} \log^{1. 5}{d})$ where $L^\star$ is the total loss of the best expert.
Differentially Private Heavy Hitter Detection using Federated Analytics
In this work, we study practical heuristics to improve the performance of prefix-tree based algorithms for differentially private heavy hitter detection.
Samplable Anonymous Aggregation for Private Federated Data Analysis
We revisit the problem of designing scalable protocols for private statistics and private federated learning when each device holds its private data.
Mean Estimation with User-level Privacy under Data Heterogeneity
In this work we propose a simple model of heterogeneous user data that allows user data to differ in both distribution and quantity of data, and provide a method for estimating the population-level mean while preserving user-level differential privacy.
Federated Learning with Differential Privacy for End-to-End Speech Recognition
($4. 5$, $10^{-9}$)-$\textbf{DP}$) with a 1. 3% (resp.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Private Vector Mean Estimation in the Shuffle Model: Optimal Rates Require Many Messages
We study the problem of private vector mean estimation in the shuffle model of privacy where $n$ users each have a unit vector $v^{(i)} \in\mathbb{R}^d$.
