no code implementations • 27 Jun 2024 • Vitaly Feldman, Audra McMillan, Satchit Sivakumar, Kunal Talwar

For distributions $P$ over $\mathbb{R}$, we consider a strong notion of instance-optimality: an algorithm that uniformly achieves the instance-optimal estimation rate is competitive with an algorithm that is told that the distribution is either $P$ or $Q_P$ for some distribution $Q_P$ whose probability density function (pdf) is within a factor of 2 of the pdf of $P$.

no code implementations • 16 Apr 2024 • Hilal Asi, Vitaly Feldman, Jelani Nelson, Huy L. Nguyen, Kunal Talwar, Samson Zhou

We study the problem of private vector mean estimation in the shuffle model of privacy where $n$ users each have a unit vector $v^{(i)} \in\mathbb{R}^d$.

no code implementations • 19 Dec 2023 • Aadirupa Saha, Vitaly Feldman, Tomer Koren, Yishay Mansour

We next study a $m$-multiway comparison (`battling') feedback, where the learner can get to see the argmin feedback of $m$-subset of queried points and show a convergence rate of $\smash{\widetilde O}(\frac{d}{ \min\{\log m, d\}\epsilon })$.

no code implementations • 29 Sep 2023 • Martin Pelikan, Sheikh Shams Azam, Vitaly Feldman, Jan "Honza" Silovsky, Kunal Talwar, Tatiana Likhomanenko

($4. 5$, $10^{-9}$)-$\textbf{DP}$) with a 1. 3% (resp.

Automatic Speech Recognition
Automatic Speech Recognition (ASR)
**+2**

no code implementations • 28 Jul 2023 • Rachel Cummings, Vitaly Feldman, Audra McMillan, Kunal Talwar

In this work we propose a simple model of heterogeneous user data that allows user data to differ in both distribution and quantity of data, and provide a method for estimating the population-level mean while preserving user-level differential privacy.

no code implementations • 27 Jul 2023 • Kunal Talwar, Shan Wang, Audra McMillan, Vojta Jina, Vitaly Feldman, Pansy Bansal, Bailey Basile, Aine Cahill, Yi Sheng Chan, Mike Chatzidakis, Junye Chen, Oliver Chick, Mona Chitnis, Suman Ganta, Yusuf Goren, Filip Granqvist, Kristine Guo, Frederic Jacobs, Omid Javidbakht, Albert Liu, Richard Low, Dan Mascenik, Steve Myers, David Park, Wonhee Park, Gianni Parsa, Tommy Pauly, Christian Priebe, Rehan Rishi, Guy Rothblum, Michael Scaria, Linmao Song, Congzheng Song, Karl Tarbe, Sebastian Vogt, Luke Winstrom, Shundong Zhou

This gap has led to significant interest in the design and implementation of simple cryptographic primitives, that can allow central-like utility guarantees without having to trust a central server.

no code implementations • 21 Jul 2023 • Karan Chadha, Junye Chen, John Duchi, Vitaly Feldman, Hanieh Hashemi, Omid Javidbakht, Audra McMillan, Kunal Talwar

In this work, we study practical heuristics to improve the performance of prefix-tree based algorithms for differentially private heavy hitter detection.

no code implementations • 27 Feb 2023 • Hilal Asi, Vitaly Feldman, Tomer Koren, Kunal Talwar

We also develop an adaptive algorithm for the small-loss setting with regret $O(L^\star\log d + \varepsilon^{-1} \log^{1. 5}{d})$ where $L^\star$ is the total loss of the best expert.

no code implementations • 24 Oct 2022 • John Duchi, Vitaly Feldman, Lunjia Hu, Kunal Talwar

Our goal is to recover the linear subspace shared by $\mu_1,\ldots,\mu_n$ using the data points from all users, where every data point from user $i$ is formed by adding an independent mean-zero noise vector to $\mu_i$.

no code implementations • 24 Oct 2022 • Hilal Asi, Vitaly Feldman, Tomer Koren, Kunal Talwar

Our lower bounds also show a separation between pure and approximate differential privacy for adaptive adversaries where the latter is necessary to achieve the non-private $O(\sqrt{T})$ regret.

no code implementations • 29 Sep 2022 • Nicholas Carlini, Vitaly Feldman, Milad Nasr

New methods designed to preserve data privacy require careful scrutiny.

no code implementations • 9 Aug 2022 • Vitaly Feldman, Audra McMillan, Kunal Talwar

Our second contribution is a new analysis of privacy amplification by shuffling.

no code implementations • 5 May 2022 • Hilal Asi, Vitaly Feldman, Kunal Talwar

We show that PrivUnit (Bhowmick et al. 2018) with optimized parameters achieves the optimal variance among a large family of locally private randomizers.

1 code implementation • 1 Mar 2022 • Vitaly Feldman, Jelani Nelson, Huy Lê Nguyen, Kunal Talwar

In many parameter settings used in practice this is a significant improvement over the $ O(n+k^2)$ computation cost that is achieved by the recent PI-RAPPOR algorithm (Feldman and Talwar; 2021).

no code implementations • NeurIPS 2021 • Vitaly Feldman, Tijana Zrnic

In this work, we give a method for tighter privacy loss accounting based on the value of a personalized privacy loss estimate for each individual in each analysis.

no code implementations • 2 Mar 2021 • Hilal Asi, Vitaly Feldman, Tomer Koren, Kunal Talwar

Stochastic convex optimization over an $\ell_1$-bounded domain is ubiquitous in machine learning applications such as LASSO but remains poorly understood when learning with differential privacy.

no code implementations • 24 Feb 2021 • Vitaly Feldman, Kunal Talwar

Here we demonstrate a general approach that, under standard cryptographic assumptions, compresses every efficient LDP algorithm with negligible loss in privacy and utility guarantees.

1 code implementation • 23 Dec 2020 • Vitaly Feldman, Audra McMillan, Kunal Talwar

As a direct corollary of our analysis we derive a simple and nearly optimal algorithm for frequency estimation in the shuffle model of privacy.

1 code implementation • 11 Dec 2020 • Gavin Brown, Mark Bun, Vitaly Feldman, Adam Smith, Kunal Talwar

Our problems are simple and fairly natural variants of the next-symbol prediction and the cluster labeling tasks.

no code implementations • NeurIPS 2021 • Vitaly Feldman, Tijana Zrnic

We consider a sequential setting in which a single dataset of individuals is used to perform adaptively-chosen analyses, while ensuring that the differential privacy loss of each participant does not exceed a pre-specified privacy budget.

1 code implementation • NeurIPS 2020 • Vitaly Feldman, Chiyuan Zhang

First, natural image and data distributions are (informally) known to be long-tailed, that is have a significant fraction of rare and atypical examples.

no code implementations • NeurIPS 2020 • Raef Bassily, Vitaly Feldman, Cristóbal Guzmán, Kunal Talwar

Our work is the first to address uniform stability of SGD on {\em nonsmooth} convex losses.

no code implementations • 10 May 2020 • Vitaly Feldman, Tomer Koren, Kunal Talwar

We also give a linear-time algorithm achieving the optimal bound on the excess loss for the strongly convex case, as well as a faster algorithm for the non-smooth case.

no code implementations • 24 Nov 2019 • Yuval Dagan, Vitaly Feldman

For $\epsilon$-differentially private prediction we give two new algorithms: one using $\tilde O(d/(\alpha^2\epsilon))$ samples and another one using $\tilde O(d^2/(\alpha\epsilon) + d/\alpha^2)$ samples.

no code implementations • 11 Nov 2019 • Yuval Dagan, Vitaly Feldman

Our main result is an exponential lower bound on the number of samples necessary to solve the standard task of learning a large-margin linear separator in the non-interactive LDP model.

no code implementations • NeurIPS 2019 • Raef Bassily, Vitaly Feldman, Kunal Talwar, Abhradeep Thakurta

A long line of existing work on private convex optimization focuses on the empirical loss and derives asymptotically tight bounds on the excess empirical loss.

no code implementations • 12 Jun 2019 • Vitaly Feldman

In our model, data is sampled from a mixture of subpopulations and our results show that memorization is necessary whenever the distribution of subpopulation frequencies is long-tailed.

no code implementations • 24 May 2019 • Vitaly Feldman, Roy Frostig, Moritz Hardt

We show a new upper bound of $\tilde O(\max\{\sqrt{k\log(n)/(mn)}, k/n\})$ on the worst-case bias that any attack can achieve in a prediction problem with $m$ classes.

no code implementations • 27 Feb 2019 • Vitaly Feldman, Jan Vondrak

Specifically, their bound on the estimation error of any $\gamma$-uniformly stable learning algorithm on $n$ samples and range in $[0, 1]$ is $O(\gamma \sqrt{n \log(1/\delta)} + \sqrt{\log(1/\delta)/n})$ with probability $\geq 1-\delta$.

no code implementations • NeurIPS 2018 • Vitaly Feldman, Jan Vondrak

Specifically, for a loss function with range bounded in $[0, 1]$, the generalization error of a $\gamma$-uniformly stable learning algorithm on $n$ samples is known to be within $O((\gamma +1/n) \sqrt{n \log(1/\delta)})$ of the empirical error with probability at least $1-\delta$.

no code implementations • 29 Nov 2018 • Úlfar Erlingsson, Vitaly Feldman, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, Abhradeep Thakurta

We study the collection of such statistics in the local differential privacy (LDP) model, and describe an algorithm whose privacy cost is polylogarithmic in the number of changes to a user's value.

no code implementations • NeurIPS 2019 • Amit Daniely, Vitaly Feldman

The only lower bound we are aware of is for PAC learning an artificial class of functions with respect to a uniform distribution (Kasiviswanathan et al. 2011).

no code implementations • 20 Aug 2018 • Vitaly Feldman, Ilya Mironov, Kunal Talwar, Abhradeep Thakurta

In addition, we demonstrate that we can achieve guarantees similar to those obtainable using the privacy-amplification-by-sampling technique in several natural settings where that technique cannot be applied.

no code implementations • 27 Mar 2018 • Cynthia Dwork, Vitaly Feldman

We demonstrate that this overhead can be avoided for the well-studied class of thresholds on a line and for a number of standard settings of convex regression.

no code implementations • NeurIPS 2018 • Blake Woodworth, Vitaly Feldman, Saharon Rosset, Nathan Srebro

The problem of handling adaptivity in data analysis, intentional or not, permeates a variety of fields, including test-set overfitting in ML challenges and the accumulation of invalid scientific discoveries.

no code implementations • 19 Dec 2017 • Vitaly Feldman, Thomas Steinke

We demonstrate that a simple and natural algorithm based on adding noise scaled to the standard deviation of the query provides our notion of stability.

no code implementations • 15 Jun 2017 • Vitaly Feldman, Thomas Steinke

We present an algorithm that estimates the expectations of $k$ arbitrary adaptively-chosen real-valued estimators using a number of samples that scales as $\sqrt{k}$.

no code implementations • 28 Feb 2017 • Vitaly Feldman, Badih Ghazi

Hence it is natural to ask whether algorithms using $k$-wise queries can solve learning problems more efficiently and by how much.

no code implementations • 20 Nov 2016 • Vitaly Feldman

We give algorithms for estimating the expectation of a given real-valued function $\phi:X\to {\bf R}$ on a sample drawn randomly from some unknown distribution $D$ over domain $X$, namely ${\bf E}_{{\bf x}\sim D}[\phi({\bf x})]$.

no code implementations • NeurIPS 2016 • Vitaly Feldman

In stochastic convex optimization the goal is to minimize a convex function $F(x) \doteq {\mathbf E}_{{\mathbf f}\sim D}[{\mathbf f}(x)]$ over a convex set $\cal K \subset {\mathbb R}^d$ where $D$ is some unknown distribution and each $f(\cdot)$ in the support of $D$ is convex over $\cal K$.

no code implementations • 7 Aug 2016 • Vitaly Feldman

We give applications of our techniques to two open problems in learning theory and to algorithms that are subject to memory and communication constraints.

no code implementations • 30 Dec 2015 • Vitaly Feldman, Cristobal Guzman, Santosh Vempala

Stochastic convex optimization, where the objective is the expectation of a random convex function, is an important and widely used method with numerous applications in machine learning, statistics, operations research and other areas.

1 code implementation • NeurIPS 2015 • Cynthia Dwork, Vitaly Feldman, Moritz Hardt, Toniann Pitassi, Omer Reingold, Aaron Roth

We also formalize and address the general problem of data reuse in adaptive data analysis.

no code implementations • 13 Apr 2015 • Vitaly Feldman, Jan Vondrak

This improves on previous approaches that all showed an upper bound of $O(1/\epsilon^2)$ for submodular and XOS functions.

no code implementations • 10 Nov 2014 • Cynthia Dwork, Vitaly Feldman, Moritz Hardt, Toniann Pitassi, Omer Reingold, Aaron Roth

We show that, surprisingly, there is a way to estimate an exponential in $n$ number of expectations accurately even if the functions are chosen adaptively.

no code implementations • 27 May 2014 • Vitaly Feldman, Pravesh Kothari

This directly gives an agnostic learning algorithm for disjunctions on symmetric distributions that runs in time $n^{O( \log{(1/\epsilon)})}$.

no code implementations • 21 May 2014 • Dana Dachman-Soled, Vitaly Feldman, Li-Yang Tan, Andrew Wan, Karl Wimmer

We study the notion of $\mathit{approximate}$ $\mathit{resilience}$ of Boolean functions, where we say that $f$ is $\alpha$-approximately $d$-resilient if $f$ is $\alpha$-close to a $[-1, 1]$-valued $d$-resilient function in $\ell_1$ distance.

no code implementations • 18 Apr 2014 • Vitaly Feldman, Pravesh Kothari, Jan Vondrák

Previous techniques considered stronger $\ell_2$ approximation and proved nearly tight bounds of $\Theta(1/\epsilon^{2})$ on the degree and $2^{\Theta(1/\epsilon^2)}$ on the number of variables.

no code implementations • 25 Feb 2014 • Vitaly Feldman, David Xiao

Our second contribution and the main tool is an equivalence between the sample complexity of (pure) differentially private learning of a concept class $C$ (or $SCDP(C)$) and the randomized one-way communication complexity of the evaluation problem for concepts from $C$.

no code implementations • NeurIPS 2013 • Maria-Florina F. Balcan, Vitaly Feldman

We describe a framework for designing efficient active learning algorithms that are tolerant to random classification noise.

no code implementations • 12 Jul 2013 • Vitaly Feldman, Jan Vondrak

This is the first algorithm in the PMAC model that over the uniform distribution can achieve a constant approximation factor arbitrarily close to 1 for all submodular functions.

no code implementations • 11 Jul 2013 • Maria Florina Balcan, Vitaly Feldman

These results combined with our generic conversion lead to the first computationally-efficient algorithms for actively learning some of these concept classes in the presence of random classification noise that provide exponential improvement in the dependence on the error $\epsilon$ over their passive counterparts.

no code implementations • 8 Apr 2013 • Vitaly Feldman, Pravesh Kothari

As an application of our learning results, we give simple differentially-private algorithms for releasing monotone conjunction counting queries with low average error.

no code implementations • 2 Apr 2013 • Vitaly Feldman, Pravesh Kothari, Jan Vondrak

We show that these structural results can be exploited to give an attribute-efficient PAC learning algorithm for submodular functions running in time $\tilde{O}(n^2) \cdot 2^{O(1/\epsilon^{4})}$.

no code implementations • 5 Nov 2012 • Pranjal Awasthi, Vitaly Feldman, Varun Kanade

We introduce a new model of membership query (MQ) learning, where the learning algorithm is restricted to query points that are \emph{close} to random examples drawn from the underlying distribution.

no code implementations • 3 Mar 2012 • Vitaly Feldman

This property is crucial for learning of DNF expressions over smoothed product distributions, a learning model introduced by Kalai et al. (2009) and inspired by the seminal smoothed analysis model of Spielman and Teng (2001).

no code implementations • 16 Feb 2010 • Vitaly Feldman

The preservation of efficiency is achieved using a new boosting technique and allows us to derive a new approach to the design of evolutionary algorithms in Valiant's (2006) model of evolvability.

Cannot find the paper you are looking for? You can
Submit a new open access paper.

Contact us on:
hello@paperswithcode.com
.
Papers With Code is a free resource with all data licensed under CC-BY-SA.