no code implementations • ICML 2020 • Daniel Rothchild, Ashwinee Panda, Enayat Ullah, Nikita Ivkin, Vladimir Braverman, Joseph Gonzalez, Ion Stoica, Raman Arora
A key insight in the design of FedSketchedSGD is that, because the Count Sketch is linear, momentum and error accumulation can both be carried out within the sketch.
no code implementations • 6 Mar 2024 • Enayat Ullah, Michael Menart, Raef Bassily, Cristóbal Guzmán, Raman Arora
We also study PA-DP supervised learning with \textit{unlabeled} public samples.
no code implementations • 22 Nov 2023 • Michael Menart, Enayat Ullah, Raman Arora, Raef Bassily, Cristóbal Guzmán
We further show that, without assuming the KL condition, the same gradient descent algorithm can achieve fast convergence to a stationary point when the gradient stays sufficiently large during the run of the algorithm.
1 code implementation • 20 Jul 2023 • Enayat Ullah, Christopher A. Choquette-Choo, Peter Kairouz, Sewoong Oh
We propose new techniques for reducing communication in private federated learning without the need for setting or tuning compression rates.
no code implementations • 20 Jul 2023 • Enayat Ullah, Raman Arora
We give efficient unlearning algorithms for linear and prefix-sum query classes.
no code implementations • 18 Jun 2022 • Yunjuan Wang, Enayat Ullah, Poorya Mianjy, Raman Arora
Recent works show that adversarial examples exist for random neural networks [Daniely and Schacham, 2020] and that these examples can be found using a single step of gradient ascent [Bubeck et al., 2021].
no code implementations • 2 Jun 2022 • Raman Arora, Raef Bassily, Tomás González, Cristóbal Guzmán, Michael Menart, Enayat Ullah
We provide a new efficient algorithm that finds an $\tilde{O}\big(\big[\frac{\sqrt{d}}{n\varepsilon}\big]^{2/3}\big)$-stationary point in the finite-sum setting, where $n$ is the number of samples.
no code implementations • 6 May 2022 • Raman Arora, Raef Bassily, Cristóbal Guzmán, Michael Menart, Enayat Ullah
For this case, we close the gap in the existing work and show that the optimal rate is (up to log factors) $\Theta\left(\frac{\Vert w^*\Vert}{\sqrt{n}} + \min\left\{\frac{\Vert w^*\Vert}{\sqrt{n\epsilon}},\frac{\sqrt{\text{rank}}\Vert w^*\Vert}{n\epsilon}\right\}\right)$, where $\text{rank}$ is the rank of the design matrix.
no code implementations • 25 Feb 2021 • Enayat Ullah, Tung Mai, Anup Rao, Ryan Rossi, Raman Arora
Our key contribution is the design of corresponding efficient unlearning algorithms, which are based on constructing a (maximal) coupling of Markov chains for the noisy SGD procedure.
no code implementations • 15 Jul 2020 • Daniel Rothchild, Ashwinee Panda, Enayat Ullah, Nikita Ivkin, Ion Stoica, Vladimir Braverman, Joseph Gonzalez, Raman Arora
A key insight in the design of FetchSGD is that, because the Count Sketch is linear, momentum and error accumulation can both be carried out within the sketch.
no code implementations • 22 Feb 2020 • Raman Arora, Teodor V. Marinov, Enayat Ullah
In this paper, we revisit the problem of private stochastic convex optimization.
2 code implementations • NeurIPS 2019 • Nikita Ivkin, Daniel Rothchild, Enayat Ullah, Vladimir Braverman, Ion Stoica, Raman Arora
Large-scale distributed training of neural networks is often limited by network bandwidth, wherein the communication time overwhelms the local computation time.
1 code implementation • 2 Aug 2018 • Enayat Ullah, Poorya Mianjy, Teodor V. Marinov, Raman Arora
We study the statistical and computational aspects of kernel principal component analysis using random Fourier features and show that under mild assumptions, $O(\sqrt{n} \log n)$ features suffices to achieve $O(1/\epsilon^2)$ sample complexity.
no code implementations • ICLR 2019 • Soham De, Anirbit Mukherjee, Enayat Ullah
Through these experiments we demonstrate the interesting sensitivity that ADAM has to its momentum parameter $\beta_1$.