Search Results for author: Badih Ghazi

Found 30 papers, 2 papers with code

The Optimality of Correlated Sampling

1 code implementation4 Dec 2016 Mohammad Bavarian, Badih Ghazi, Elad Haramaty, Pritish Kamath, Ronald L. Rivest, Madhu Sudan

In this note, we give a surprisingly simple proof that this protocol is in fact tight.

Computational Complexity Information Theory Information Theory

On the Power of Learning from $k$-Wise Queries

no code implementations28 Feb 2017 Vitaly Feldman, Badih Ghazi

Hence it is natural to ask whether algorithms using $k$-wise queries can solve learning problems more efficiently and by how much.

PAC learning

Recursive Sketches for Modular Deep Learning

no code implementations29 May 2019 Badih Ghazi, Rina Panigrahy, Joshua R. Wang

The sketch summarizes essential information about the inputs and outputs of the network and can be used to quickly identify key components and summary statistics of the inputs.

Scalable and Differentially Private Distributed Aggregation in the Shuffled Model

no code implementations19 Jun 2019 Badih Ghazi, Rasmus Pagh, Ameya Velingker

Federated learning promises to make machine learning feasible on distributed, private datasets by implementing gradient descent using secure aggregation methods.

Federated Learning Privacy Preserving

On the Power of Multiple Anonymous Messages

no code implementations29 Aug 2019 Badih Ghazi, Noah Golowich, Ravi Kumar, Rasmus Pagh, Ameya Velingker

- Protocols in the multi-message shuffled model with $poly(\log{B}, \log{n})$ bits of communication per user and $poly\log{B}$ error, which provide an exponential improvement on the error compared to what is possible with single-message algorithms.

Private Aggregation from Fewer Anonymous Messages

no code implementations24 Sep 2019 Badih Ghazi, Pasin Manurangsi, Rasmus Pagh, Ameya Velingker

Using a reduction of Balle et al. (2019), our improved analysis of the protocol of Ishai et al. yields, in the same model, an $\left(\varepsilon, \delta\right)$-differentially private protocol for aggregation that, for any constant $\varepsilon > 0$ and any $\delta = \frac{1}{\mathrm{poly}(n)}$, incurs only a constant error and requires only a constant number of messages per party.

Cryptography and Security Data Structures and Algorithms

Near-tight closure bounds for Littlestone and threshold dimensions

no code implementations7 Jul 2020 Badih Ghazi, Noah Golowich, Ravi Kumar, Pasin Manurangsi

We study closure properties for the Littlestone and threshold dimensions of binary hypothesis classes.

Differentially Private Clustering: Tight Approximation Ratios

no code implementations NeurIPS 2020 Badih Ghazi, Ravi Kumar, Pasin Manurangsi

For several basic clustering problems, including Euclidean DensestBall, 1-Cluster, k-means, and k-median, we give efficient differentially private algorithms that achieve essentially the same approximation ratios as those that can be obtained by any non-private algorithm, while incurring only small additive errors.

Clustering

On Distributed Differential Privacy and Counting Distinct Elements

no code implementations21 Sep 2020 Lijie Chen, Badih Ghazi, Ravi Kumar, Pasin Manurangsi

We study the setup where each of $n$ users holds an element from a discrete set, and the goal is to count the number of distinct elements across all users, under the constraint of $(\epsilon, \delta)$-differentially privacy: - In the non-interactive local setting, we prove that the additive error of any protocol is $\Omega(n)$ for any constant $\epsilon$ and for any $\delta$ inverse polynomial in $n$.

Open-Ended Question Answering

Robust and Private Learning of Halfspaces

no code implementations30 Nov 2020 Badih Ghazi, Ravi Kumar, Pasin Manurangsi, Thao Nguyen

In this work, we study the trade-off between differential privacy and adversarial robustness under L2-perturbations in the context of learning halfspaces.

Adversarial Robustness

Sample-efficient proper PAC learning with approximate differential privacy

no code implementations7 Dec 2020 Badih Ghazi, Noah Golowich, Ravi Kumar, Pasin Manurangsi

In this paper we prove that the sample complexity of properly learning a class of Littlestone dimension $d$ with approximate differential privacy is $\tilde O(d^6)$, ignoring privacy and accuracy parameters.

PAC learning

On Avoiding the Union Bound When Answering Multiple Differentially Private Queries

no code implementations16 Dec 2020 Badih Ghazi, Ravi Kumar, Pasin Manurangsi

On the other hand, the algorithm of Dagan and Kur has a remarkable advantage that the $\ell_{\infty}$ error bound of $O(\frac{1}{\epsilon}\sqrt{k \log \frac{1}{\delta}})$ holds not only in expectation but always (i. e., with probability one) while we can only get a high probability (or expected) guarantee on the error.

Deep Learning with Label Differential Privacy

no code implementations NeurIPS 2021 Badih Ghazi, Noah Golowich, Ravi Kumar, Pasin Manurangsi, Chiyuan Zhang

The Randomized Response (RR) algorithm is a classical technique to improve robustness in survey aggregation, and has been widely adopted in applications with differential privacy guarantees.

Multi-class Classification

Locally Private k-Means in One Round

no code implementations20 Apr 2021 Alisa Chang, Badih Ghazi, Ravi Kumar, Pasin Manurangsi

We provide an approximation algorithm for k-means clustering in the one-round (aka non-interactive) local model of differential privacy (DP).

Clustering Open-Ended Question Answering

Large-Scale Differentially Private BERT

no code implementations3 Aug 2021 Rohan Anil, Badih Ghazi, Vineet Gupta, Ravi Kumar, Pasin Manurangsi

In this work, we study the large-scale pretraining of BERT-Large with differentially private SGD (DP-SGD).

Language Modelling

User-Level Private Learning via Correlated Sampling

no code implementations21 Oct 2021 Badih Ghazi, Ravi Kumar, Pasin Manurangsi

Most works in learning with differential privacy (DP) have focused on the setting where each user has a single sample.

User-Level Differentially Private Learning via Correlated Sampling

no code implementations NeurIPS 2021 Badih Ghazi, Ravi Kumar, Pasin Manurangsi

Most works in learning with differential privacy (DP) have focused on the setting where each user has a single sample.

Faster Privacy Accounting via Evolving Discretization

no code implementations10 Jul 2022 Badih Ghazi, Pritish Kamath, Ravi Kumar, Pasin Manurangsi

We introduce a new algorithm for numerical composition of privacy random variables, useful for computing the accurate differential privacy parameters for composition of mechanisms.

Connect the Dots: Tighter Discrete Approximations of Privacy Loss Distributions

no code implementations10 Jul 2022 Vadym Doroshenko, Badih Ghazi, Pritish Kamath, Ravi Kumar, Pasin Manurangsi

The privacy loss distribution (PLD) provides a tight characterization of the privacy loss of a mechanism in the context of differential privacy (DP).

Algorithms with More Granular Differential Privacy Guarantees

no code implementations8 Sep 2022 Badih Ghazi, Ravi Kumar, Pasin Manurangsi, Thomas Steinke

Differential privacy is often applied with a privacy parameter that is larger than the theory suggests is ideal; various informal justifications for tolerating large privacy parameters have been proposed.

Attribute

Anonymized Histograms in Intermediate Privacy Models

no code implementations27 Oct 2022 Badih Ghazi, Pritish Kamath, Ravi Kumar, Pasin Manurangsi

We study the problem of privately computing the anonymized histogram (a. k. a.

Private Isotonic Regression

no code implementations27 Oct 2022 Badih Ghazi, Pritish Kamath, Ravi Kumar, Pasin Manurangsi

For the most general problem of isotonic regression over a partially ordered set (poset) $\mathcal{X}$ and for any Lipschitz loss function, we obtain a pure-DP algorithm that, given $n$ input points, has an expected excess empirical risk of roughly $\mathrm{width}(\mathcal{X}) \cdot \log|\mathcal{X}| / n$, where $\mathrm{width}(\mathcal{X})$ is the width of the poset.

regression

Private Ad Modeling with DP-SGD

no code implementations21 Nov 2022 Carson Denison, Badih Ghazi, Pritish Kamath, Ravi Kumar, Pasin Manurangsi, Krishna Giri Narra, Amer Sinha, Avinash V Varadarajan, Chiyuan Zhang

A well-known algorithm in privacy-preserving ML is differentially private stochastic gradient descent (DP-SGD).

Privacy Preserving

Regression with Label Differential Privacy

no code implementations12 Dec 2022 Badih Ghazi, Pritish Kamath, Ravi Kumar, Ethan Leeman, Pasin Manurangsi, Avinash V Varadarajan, Chiyuan Zhang

We study the task of training regression models with the guarantee of label differential privacy (DP).

regression

On User-Level Private Convex Optimization

no code implementations8 May 2023 Badih Ghazi, Pritish Kamath, Ravi Kumar, Raghu Meka, Pasin Manurangsi, Chiyuan Zhang

We introduce a new mechanism for stochastic convex optimization (SCO) with user-level differential privacy guarantees.

Ticketed Learning-Unlearning Schemes

no code implementations27 Jun 2023 Badih Ghazi, Pritish Kamath, Ravi Kumar, Pasin Manurangsi, Ayush Sekhari, Chiyuan Zhang

Subsequently, given any subset of examples that wish to be unlearnt, the goal is to learn, without the knowledge of the original training dataset, a good predictor that is identical to the predictor that would have been produced when learning from scratch on the surviving examples.

Training Differentially Private Ad Prediction Models with Semi-Sensitive Features

no code implementations26 Jan 2024 Lynn Chua, Qiliang Cui, Badih Ghazi, Charlie Harrison, Pritish Kamath, Walid Krichene, Ravi Kumar, Pasin Manurangsi, Krishna Giri Narra, Amer Sinha, Avinash Varadarajan, Chiyuan Zhang

Motivated by problems arising in digital advertising, we introduce the task of training differentially private (DP) machine learning models with semi-sensitive features.

How Private is DP-SGD?

no code implementations26 Mar 2024 Lynn Chua, Badih Ghazi, Pritish Kamath, Ravi Kumar, Pasin Manurangsi, Amer Sinha, Chiyuan Zhang

We demonstrate a substantial gap between the privacy guarantees of the Adaptive Batch Linear Queries (ABLQ) mechanism under different types of batch sampling: (i) Shuffling, and (ii) Poisson subsampling; the typical analysis of Differentially Private Stochastic Gradient Descent (DP-SGD) follows by interpreting it as a post-processing of ABLQ.

Cannot find the paper you are looking for? You can Submit a new open access paper.