Search Results for author:
Found 12 papers, 10 papers with code
LLMail-Inject: A Dataset from a Realistic Adaptive Prompt Injection Challenge
Indirect Prompt Injection attacks exploit the inherent limitation of Large Language Models (LLMs) to distinguish between instructions and data in their inputs.
Dataset and Lessons Learned from the 2024 SaTML LLM Capture-the-Flag Competition
To study this problem, we organized a capture-the-flag competition at IEEE SaTML 2024, where the flag is a secret string in the LLM system prompt.
Get my drift? Catching LLM Task Drift with Activation Deltas
We study LLM activations as a solution to detect task drift, showing that activation deltas - the difference in activations before and after processing external data - are strongly correlated with this phenomenon.
Closed-Form Bounds for DP-SGD against Record-level Inference
This paper presents a new approach to evaluate the privacy of machine learning models against specific record-level threats, such as membership and attribute inference, without the indirection through DP.
SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learning
Deploying machine learning models in production may allow adversaries to infer sensitive information about training data.
Synthetic Data -- what, why and how?
This explainer document aims to provide an overview of the current state of the rapidly expanding work on synthetic data technologies, with a particular focus on privacy.
Approximating Full Conformal Prediction at Scale via Influence Functions
We prove that our method is a consistent approximation of full CP, and empirically show that the approximation error becomes smaller as the training set increases; e. g., for $10^{3}$ training points the two methods output p-values that are $<10^{-3}$ apart: a negligible error for any practical application.
Reconstructing Training Data with Informed Adversaries
Our work provides an effective reconstruction attack that model developers can use to assess memorization of individual points in general settings beyond those considered in previous works (e. g. generative language models or access to training gradients); it shows that standard models have the capacity to store enough information to enable high-fidelity reconstruction of training data points; and it demonstrates that differential privacy can successfully mitigate such attacks in a parameter regime where utility degradation is minimal.
Exact Optimization of Conformal Predictors via Incremental and Decremental Learning
We evaluate our findings empirically, and discuss when methods are suitable for CP optimization.
Disparate Vulnerability to Membership Inference Attacks
Differential privacy bounds disparate vulnerability but can significantly reduce the accuracy of the model.
F-BLEAU: Fast Black-box Leakage Estimation
The state-of-the-art method for estimating these leakage measures is the frequentist paradigm, which approximates the system's internals by looking at the frequencies of its inputs and outputs.
Cryptography and Security
Bayes, not Naïve: Security Bounds on Website Fingerprinting Defenses
In this paper, we present a practical method to derive security bounds for any WF defense, which depend on a chosen feature set.
Cryptography and Security
