Search Results for author:
Found 44 papers, 14 papers with code
Cascading Adversarial Bias from Injection to Distillation in Language Models
With only 25 poisoned samples (0. 25% poisoning rate), student models generate biased responses 76. 9% of the time in targeted scenarios - higher than 69. 4% in teacher models.
Strong Membership Inference Attacks on Massive Datasets and (Moderately) Large Language Models
State-of-the-art membership inference attacks (MIAs) typically require training many reference models, making it difficult to scale these attacks to large pre-trained language models (LLMs).
Lessons from Defending Gemini Against Indirect Prompt Injections
Gemini is increasingly used to perform tasks on behalf of users, where function-calling and tool-use capabilities enable the model to access user data.
Defeating Prompt Injections by Design
Large Language Models (LLMs) are increasingly deployed in agentic systems that interact with an external environment.
Large Language Models Can Verbatim Reproduce Long Malicious Sequences
This paper re-examines the concept of backdoor attacks in the context of Large Language Models (LLMs), focusing on the generation of long, verbatim sequences.
$(\varepsilon, δ)$ Considered Harmful: Best Practices for Reporting Differential Privacy Guarantees
Current practices for reporting the level of differential privacy (DP) guarantees for machine learning (ML) algorithms provide an incomplete and potentially misleading picture of the guarantees and make it difficult to compare privacy levels across different settings.
Interpreting the Repeated Token Phenomenon in Large Language Models
Large Language Models (LLMs), despite their impressive capabilities, often fail to accurately repeat a single word when prompted to, and instead output unrelated text.
Machine Unlearning Doesn't Do What You Think: Lessons for Generative AI Policy, Research, and Practice
We articulate fundamental mismatches between technical methods for machine unlearning in Generative AI, and documented aspirations for broader impact that these methods could have for law and policy.
To Shuffle or not to Shuffle: Auditing DP-SGD with Shuffling
At the same time, we do not know how to compute tight theoretical guarantees for shuffling; thus, DP guarantees of models privately trained with shuffling are often reported as though Poisson sub-sampling was used.
Stealing User Prompts from Mixture of Experts
Mixture-of-Experts (MoE) models improve the efficiency and scalability of dense language models by routing each token to a small number of experts in each layer.
Measuring memorization through probabilistic discoverable extraction
Large language models (LLMs) are susceptible to memorizing training data, raising concerns due to the potential extraction of sensitive information.
The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGD
The standard composition-based privacy analysis of DP-SGD effectively assumes that the adversary has access to all intermediate iterates, which is often unrealistic.
UnUnlearning: Unlearning is not sufficient for content regulation in advanced generative AI
The promise is that if the model does not have a certain malicious capability, then it cannot be used for the associated malicious purpose.
Measuring memorization in RLHF for code completion
In contrast, we find that aligning by learning directly from human preference data via a special case of $\Psi$PO, Identity Preference Optimization (IPO), increases the likelihood that training data is regurgitated compared to RLHF.
Beyond Slow Signs in High-fidelity Model Extraction
Our study evaluates the feasibility of parameter extraction methods of Carlini et al. [1] further enhanced by Canales-Mart\'inez et al. [2] for models trained on standard benchmarks.
Are we making progress in unlearning? Findings from the first NeurIPS unlearning competition
We present the findings of the first NeurIPS competition on unlearning, which sought to stimulate the development of novel algorithms and initiate discussions on formal and robust evaluation methodologies.
Beyond the Calibration Point: Mechanism Comparison in Differential Privacy
In differentially private (DP) machine learning, the privacy guarantees of DP mechanisms are often reported and compared on the basis of a single $(\varepsilon, \delta)$-pair.
Locking Machine Learning Models into Hardware
We demonstrate that \emph{locking} mechanisms are feasible by either targeting efficiency of model representations, making such models incompatible with quantization, or tying the model's operation to specific characteristics of hardware, such as the number of clock cycles for arithmetic operations.
Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy
In the privacy literature, this is known as membership inference.
Buffer Overflow in Mixture of Experts
Mixture of Experts (MoE) has become a key ingredient for scaling large foundation models while keeping inference costs steady.
Unlocking Accuracy and Fairness in Differentially Private Image Classification
The poor performance of classifiers trained with DP has prevented the widespread adoption of privacy preserving machine learning in industry.
Bounding data reconstruction attacks with the hypothesis testing interpretation of differential privacy
We explore Reconstruction Robustness (ReRo), which was recently proposed as an upper bound on the success of data reconstruction attacks against machine learning models.
Differentially Private Diffusion Models Generate Useful Synthetic Images
By privately fine-tuning ImageNet pre-trained diffusion models with more than 80M parameters, we obtain SOTA results on CIFAR-10 and Camelyon17 in terms of both FID and the accuracy of downstream classifiers trained on synthetic data.
Towards Unbounded Machine Unlearning
This paper is the first, to our knowledge, to study unlearning for different applications (RB, RC, UP), with the view that each has its own desiderata, definitions for `forgetting' and associated metrics for forget quality.
Tight Auditing of Differentially Private Machine Learning
Moreover, our auditing scheme requires only two training runs (instead of thousands) to produce tight privacy estimates, by adapting recent advances in tight composition theorems for differential privacy.
Extracting Training Data from Diffusion Models
Image diffusion models such as DALL-E 2, Imagen, and Stable Diffusion have attracted significant attention due to their ability to generate high-quality synthetic images.
Unlocking High-Accuracy Differentially Private Image Classification through Scale
Differential Privacy (DP) provides a formal privacy guarantee preventing adversaries with access to a machine learning model from extracting information about individual training points.
Reconstructing Training Data with Informed Adversaries
Our work provides an effective reconstruction attack that model developers can use to assess memorization of individual points in general settings beyond those considered in previous works (e. g. generative language models or access to training gradients); it shows that standard models have the capacity to store enough information to enable high-fidelity reconstruction of training data points; and it demonstrates that differential privacy can successfully mitigate such attacks in a parameter regime where utility degradation is minimal.
Learning to be adversarially robust and differentially private
We study the difficulties in learning that arise from robust and differentially private optimization.
Towards transformation-resilient provenance detection of digital media
In this paper, we introduce ReSWAT (Resilient Signal Watermarking via Adversarial Training), a framework for learning transformation-resilient watermark detectors that are able to detect a watermark even after a signal has been through several post-processing transformations.
Adaptive Webpage Fingerprinting from TLS Traces
In webpage fingerprinting, an on-path adversary infers the specific webpage loaded by a victim user by analysing the patterns in the encrypted TLS traffic exchanged between the user's browser and the website's servers.
Local and Central Differential Privacy for Robustness and Privacy in Federated Learning
This paper investigates whether and to what extent one can use differential Privacy (DP) to protect both privacy and robustness in FL.
Trade-offs between membership privacy & adversarially robust learning
Consequently, an abundance of research has been devoted to designing machine learning methods that are robust to adversarial examples.
Extensions and limitations of randomized smoothing for robustness guarantees
Randomized smoothing, a method to certify a classifier's decision on an input is invariant under adversarial noise, offers attractive advantages over other certification methods.
Unique properties of adversarially trained linear classifiers on Gaussian data
Machine learning models are vulnerable to adversarial perturbations, that when added to an input, can cause high confidence misclassifications.
A FRAMEWORK FOR ROBUSTNESS CERTIFICATION OF SMOOTHED CLASSIFIERS USING F-DIVERGENCES
Formal verification techniques that compute provable guarantees on properties of machine learning models, like robustness to norm-bounded adversarial perturbations, have yielded impressive results.
Provenance detection through learning transformation-resilient watermarking
In this paper, we introduce ReSWAT (Resilient Signal Watermarking via Adversarial Training), a framework for learning transformation-resilient watermark detectors that are able to detect a watermark even after a signal has been through several post-processing transformations.
Contamination Attacks and Mitigation in Multi-Party Machine Learning
Machine learning is data hungry; the more data a model has access to in training, the more likely it is to perform well at inference time.
A note on hyperparameters in black-box adversarial examples
Black-box attacks assume no knowledge of the model weights or architecture.
Evading classifiers in discrete domains with provable optimality guarantees
We introduce a graphical framework that (1) generalizes existing attacks in discrete domains, (2) can accommodate complex cost functions beyond $p$-norms, including financial cost incurred when attacking a classifier, and (3) efficiently produces valid adversarial examples with guarantees of minimal adversarial cost.
Learning Universal Adversarial Perturbations with Generative Models
Neural networks are known to be vulnerable to adversarial examples, inputs that have been intentionally perturbed to remain visually similar to the source input, but cause a misclassification.
Ranked #9 on
Graph Classification
on NCI1
(using extra training data)
LOGAN: Membership Inference Attacks Against Generative Models
Generative models estimate the underlying distribution of a dataset to generate realistic samples according to that distribution.
Generating Steganographic Images via Adversarial Training
In this paper, we apply adversarial training techniques to the discriminative task of learning a steganographic algorithm.
