Search Results for author:
Found 14 papers, 5 papers with code
Privacy Leakage in Text Classification A Data Extraction Approach
Recent work has demonstrated the successful extraction of training data from generative language models.
Differentially Private Training of Mixture of Experts Models
This position paper investigates the integration of Differential Privacy (DP) in the training of Mixture of Experts (MoE) models within the field of natural language processing.
Privately Aligning Language Models with Reinforcement Learning
Positioned between pre-training and user deployment, aligning large language models (LLMs) through reinforcement learning (RL) has emerged as a prevailing strategy for training instruction following-models such as ChatGPT.
Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generation
Our results demonstrate that our algorithm can achieve competitive performance with strong privacy levels.
Planting and Mitigating Memorized Content in Predictive-Text Language Models
Language models are widely deployed to provide automatic text completion services in user products.
Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe
Privacy concerns have attracted increasing attention in data-driven products due to the tendency of machine learning models to memorize sensitive training data.
When Does Differentially Private Learning Not Suffer in High Dimensions?
Large pretrained models can be privately fine-tuned to achieve performance approaching that of non-private models.
Privacy Leakage in Text Classification: A Data Extraction Approach
Recent work has demonstrated the successful extraction of training data from generative language models.
Differentially Private Fine-tuning of Language Models
For example, on the MNLI dataset we achieve an accuracy of $87. 8\%$ using RoBERTa-Large and $83. 5\%$ using RoBERTa-Base with a privacy budget of $\epsilon = 6. 7$.
Membership Inference on Word Embedding and Beyond
Indeed, our attack is a cheaper membership inference attack on text-generative models, which does not require the knowledge of the target model or any expensive training of text-generative models as shadow models.
Privacy Regularization: Joint Privacy-Utility Optimization in Language Models
In this work, we introduce two privacy-preserving regularization methods for training language models that enable joint optimization of utility and privacy through (1) the use of a discriminator and (2) the inclusion of a triplet-loss term.
Training Data Leakage Analysis in Language Models
It has been demonstrated that strong performance of language models comes along with the ability to memorize rare training samples, which poses serious privacy threats in case the model is trained on confidential user content.
rTop-k: A Statistical Estimation Approach to Distributed SGD
The statistically optimal communication scheme arising from the analysis of this model leads to a new sparsification technique for SGD, which concatenates random-k and top-k, considered separately in the prior literature.
Improving Semantic Parsing with Neural Generator-Reranker Architecture
In this work, we propose a generator-reranker architecture for semantic parsing.
