Search Results for author:
Found 16 papers, 10 papers with code
Differentially Private Synthetic Data via Foundation Model APIs 2: Text
Lin et al. (2024) recently introduced the Private Evolution (PE) algorithm to generate DP synthetic images with only API access to diffusion models.
Privacy-Preserving Instructions for Aligning Large Language Models
Service providers of large language model (LLM) applications collect user instructions in the wild and use them in further aligning LLMs with users' intentions.
Selective Pre-training for Private Fine-tuning
Besides performance improvements, our framework also shows that with careful pre-training and private fine-tuning, smaller models can match the performance of much larger models that do not have access to private data, highlighting the promise of private learning as a tool for model compression and efficiency.
Exploring the Limits of Differentially Private Deep Learning with Group-wise Clipping
To reduce the compute time overhead of private learning, we show that \emph{per-layer clipping}, where the gradient of each neural network layer is clipped separately, allows clipping to be performed in conjunction with backpropagation in differentially private optimization.
Adversarial Noises Are Linearly Separable for (Nearly) Random Neural Networks
Adversarial examples, which are usually generated for specific inputs with a specific model, are ubiquitous for neural networks.
Individual Privacy Accounting for Differentially Private Stochastic Gradient Descent
Differentially private stochastic gradient descent (DP-SGD) is the workhorse algorithm for recent advances in private deep learning.
Availability Attacks Create Shortcuts
We are the first to unveil an important population property of the perturbations of these attacks: they are almost \textbf{linearly separable} when assigned with the target labels of the corresponding samples, which hence can work as \emph{shortcuts} for the learning objective.
Differentially Private Fine-tuning of Language Models
For example, on the MNLI dataset we achieve an accuracy of $87. 8\%$ using RoBERTa-Large and $83. 5\%$ using RoBERTa-Base with a privacy budget of $\epsilon = 6. 7$.
Large Scale Private Learning via Low-rank Reparametrization
We propose a reparametrization scheme to address the challenges of applying differentially private SGD on large neural networks, which are 1) the huge memory cost of storing individual gradients, 2) the added noise suffering notorious dimensional dependence.
Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning
The privacy leakage of the model about the training data can be bounded in the differential privacy mechanism.
On the Stability of Multi-branch Network
More importantly, we propose a new design ``STAM aggregation" that can guarantee to STAbilize the forward/backward process of Multi-branch networks irrespective of the number of branches.
How Does Data Augmentation Affect Privacy in Machine Learning?
Even further, we show that the proposed approach can achieve higher MI attack success rates on models trained with some data augmentation than the existing methods on models trained without data augmentation.
Gradient Perturbation is Underrated for Differentially Private Convex Optimization
By using the \emph{expected curvature}, we show that gradient perturbation can achieve a significantly improved utility guarantee that can theoretically justify the advantage of gradient perturbation over other perturbation methods.
VisDrone-DET2019: The Vision Meets Drone Object Detection in Image Challenge Results
Results of 33 object detection algorithms are presented.
STABILITY AND CONVERGENCE THEORY FOR LEARNING RESNET: A FULL CHARACTERIZATION
We show that for standard initialization used in practice, $\tau =1/\Omega(\sqrt{L})$ is a sharp value in characterizing the stability of forward/backward process of ResNet, where $L$ is the number of residual blocks.
Stabilize Deep ResNet with A Sharp Scaling Factor $τ$
Moreover, for ResNets with normalization layer, adding such a factor $\tau$ also stabilizes the training and obtains significant performance gain for deep ResNet.
