Search Results for author:
Found 16 papers, 2 papers with code
User-level Differentially Private Stochastic Convex Optimization: Efficient Algorithms with Optimal Rates
We study differentially private stochastic convex optimization (DP-SCO) under user-level privacy, where each user may hold multiple data items.
Detecting Pretraining Data from Large Language Models
Min-K% Prob can be applied without any knowledge about the pretraining corpus or any additional training, departing from previous detection methods that require training a reference model on data that is similar to the pretraining data.
Learning across Data Owners with Joint Differential Privacy
In this paper, we study the setting in which data owners train machine learning models collaboratively under a privacy notion called joint differential privacy [Kearns et al., 2018].
$k$NN-Adapter: Efficient Domain Adaptation for Black-Box Language Models
Fine-tuning a language model on a new domain is standard practice for domain adaptation.
Algorithmic Aspects of the Log-Laplace Transform and a Non-Euclidean Proximal Sampler
The development of efficient sampling algorithms catering to non-Euclidean geometries has been a challenging endeavor, as discretization techniques which succeed in the Euclidean setting do not readily carry over to more general settings.
ReSQueing Parallel and Private Stochastic Convex Optimization
We give a parallel algorithm obtaining optimization error $\epsilon_{\text{opt}}$ with $d^{1/3}\epsilon_{\text{opt}}^{-2/3}$ gradient oracle query depth and $d^{1/3}\epsilon_{\text{opt}}^{-2/3} + \epsilon_{\text{opt}}^{-2}$ gradient queries in total, assuming access to a bounded-variance stochastic gradient estimator.
Augmentation with Projection: Towards an Effective and Efficient Data Augmentation Paradigm for Distillation
However, these data augmentation methods either potentially cause shifts in decision boundaries (representation interpolation), are not expressive enough (token replacement), or introduce too much computational overhead (augmentation with models).
Private Convex Optimization in General Norms
We propose a new framework for differentially private optimization of convex functions which are Lipschitz in an arbitrary norm $\|\cdot\|$.
When Does Differentially Private Learning Not Suffer in High Dimensions?
Large pretrained models can be privately fine-tuned to achieve performance approaching that of non-private models.
Private Convex Optimization via Exponential Mechanism
Furthermore, we show how to implement this mechanism using $\widetilde{O}(n \min(d, n))$ queries to $f_i(x)$ for the DP-SCO where $n$ is the number of samples/users and $d$ is the ambient dimension.
Better Private Algorithms for Correlation Clustering
In machine learning, correlation clustering is an important problem whose goal is to partition the individuals into groups that correlate with their pairwise similarities as much as possible.
Private Non-smooth ERM and SCO in Subquadratic Steps
We study the differentially private Empirical Risk Minimization (ERM) and Stochastic Convex Optimization (SCO) problems for non-smooth convex functions.
Tight lower bounds for Differentially Private ERM
We consider the lower bounds of differentially private ERM for general convex functions.
The Convergence Rate of SGD's Final Iterate: Analysis on Dimension Dependence
The best known lower bounds, however, are worse than the upper bounds by a factor of $\log T$.
Lower Bounds for Differentially Private ERM: Unconstrained and Non-Euclidean
We consider the lower bounds of differentially private empirical risk minimization (DP-ERM) for convex functions in constrained/unconstrained cases with respect to the general $\ell_p$ norm beyond the $\ell_2$ norm considered by most of the previous works.
Private Non-smooth Empirical Risk Minimization and Stochastic Convex Optimization in Subquadratic Steps
More precisely, our differentially private algorithm requires $O(\frac{N^{3/2}}{d^{1/8}}+ \frac{N^2}{d})$ gradient queries for optimal excess empirical risk, which is achieved with the help of subsampling and smoothing the function via convolution.
