Search Results for author:
Found 75 papers, 38 papers with code
Privacy Side Channels in Machine Learning Systems
Most current approaches for protecting privacy in machine learning (ML) assume that models exist in a vacuum, when in reality, ML models are part of larger systems that include components for training data filtering, output monitoring, and more.
Reverse-Engineering Decoding Strategies Given Blackbox Access to a Language Generation System
Neural language models are increasingly deployed into APIs and websites that allow a user to pass in a prompt and receive generated text.
Identifying and Mitigating the Security Risks of Generative AI
However, GenAI can be used just as well by attackers to generate new attacks and increase the velocity and efficacy of existing attacks.
A LLM Assisted Exploitation of AI-Guardian
Large language models (LLMs) are now highly capable at a diverse range of tasks.
Are aligned neural networks adversarially aligned?
We show that existing NLP-based optimization attacks are insufficiently powerful to reliably attack aligned text models: even when current NLP-based attacks fail, we can find adversarial inputs with brute force.
Evading Black-box Classifiers Without Breaking Eggs
We then design new attacks that reduce the number of bad queries by $1. 5$-$7. 3\times$, but often at a significant increase in total (non-bad) queries.
Students Parrot Their Teachers: Membership Inference on Model Distillation
We explain the success of our attacks on distillation by showing that membership inference attacks on a private dataset can succeed even if the target model is *never* queried on any actual training points, but only on inputs whose predictions are highly influenced by training data.
Randomness in ML Defenses Helps Persistent Attackers and Hinders Evaluators
It is becoming increasingly imperative to design robust ML defenses.
Poisoning Web-Scale Training Datasets is Practical
Deep learning models are often trained on distributed, webscale datasets crawled from the internet.
Tight Auditing of Differentially Private Machine Learning
Moreover, our auditing scheme requires only two training runs (instead of thousands) to produce tight privacy estimates, by adapting recent advances in tight composition theorems for differential privacy.
Effective Robustness against Natural Distribution Shifts for Models with Different Training Data
In this paper, we propose a new effective robustness evaluation metric to compare the effective robustness of models trained on different data distributions.
Extracting Training Data from Diffusion Models
Image diffusion models such as DALL-E 2, Imagen, and Stable Diffusion have attracted significant attention due to their ability to generate high-quality synthetic images.
Publishing Efficient On-device Models Increases Adversarial Vulnerability
We then show that the vulnerability increases as the similarity between a full-scale and its efficient model increase.
Considerations for Differentially Private Learning with Large-Scale Public Pretraining
The performance of differentially private machine learning can be boosted significantly by leveraging the transfer learning capabilities of non-private models pretrained on large public datasets.
Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy
Studying data memorization in neural language models helps us understand the risks (e. g., to privacy or copyright) associated with models regurgitating training data and aids in the development of countermeasures.
Preprocessors Matter! Realistic Decision-Based Attacks on Machine Learning Systems
Decision-based attacks construct adversarial examples against a machine learning (ML) model by making only hard-label queries.
No Free Lunch in "Privacy for Free: How does Dataset Condensation Help Privacy"
New methods designed to preserve data privacy require careful scrutiny.
Part-Based Models Improve Adversarial Robustness
We show that combining human prior knowledge with end-to-end learning can improve the robustness of deep neural networks by introducing a part-based model for object classification.
Measuring Forgetting of Memorized Training Examples
In memorization, models overfit specific training examples and become susceptible to privacy attacks.
Increasing Confidence in Adversarial Robustness Evaluations
Hundreds of defenses have been proposed to make deep neural networks robust against minimal (adversarial) input perturbations.
The Privacy Onion Effect: Memorization is Relative
Machine learning models trained on private datasets have been shown to leak their private data.
(Certified!!) Adversarial Robustness for Free!
In this paper we show how to achieve state-of-the-art certified adversarial robustness to 2-norm bounded perturbations by relying exclusively on off-the-shelf pretrained models.
Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets
We show that an adversary who can poison a training dataset can cause models trained on this dataset to leak significant private details of training points belonging to other parties.
Debugging Differential Privacy: A Case Study for Privacy Auditing
Differential Privacy can provide provable privacy guarantees for training data in machine learning.
Quantifying Memorization Across Neural Language Models
Large language models (LMs) have been shown to memorize parts of their training data, and when prompted appropriately, they will emit the memorized training data verbatim.
Counterfactual Memorization in Neural Language Models
Modern neural language models widely used in tasks across NLP risk memorizing sensitive information from their training data.
Membership Inference Attacks From First Principles
A membership inference attack allows an adversary to query a trained machine learning model to predict whether or not a particular example was contained in the model's training dataset.
Unsolved Problems in ML Safety
Machine learning (ML) systems are rapidly increasing in size, are acquiring new capabilities, and are increasingly deployed in high-stakes settings.
Deduplicating Training Data Makes Language Models Better
As a result, over 1% of the unprompted output of language models trained on these datasets is copied verbatim from the training data.
Data Poisoning Won't Save You From Facial Recognition
We demonstrate that this strategy provides a false sense of security, as it ignores an inherent asymmetry between the parties: users' pictures are perturbed once and for all before being published (at which point they are scraped) and must thereafter fool all future models -- including models trained adaptively against the users' past attacks, or models that use technologies discovered after the attack.
Evading Adversarial Example Detection Defenses with Orthogonal Projected Gradient Descent
Evading adversarial example detection defenses requires finding adversarial examples that must simultaneously (a) be misclassified by the model and (b) be detected as non-adversarial.
Indicators of Attack Failure: Debugging and Improving Optimization of Adversarial Examples
Evaluating robustness of machine-learning models to adversarial examples is a challenging problem.
Poisoning and Backdooring Contrastive Learning
Multimodal contrastive learning methods like CLIP train on noisy and uncurated training datasets.
AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation
We extend semi-supervised learning to the problem of domain adaptation to learn significantly higher-accuracy models that train on one data distribution and test on a different one.
Semi-supervised Domain Adaptation
Unsupervised Domain Adaptation
Handcrafted Backdoors in Deep Neural Networks
When machine learning training is outsourced to third parties, $backdoor$ $attacks$ become practical as the third party who trains the model may act maliciously to inject hidden behaviors into the otherwise accurate model.
Poisoning the Unlabeled Dataset of Semi-Supervised Learning
Our attacks are highly effective across datasets and semi-supervised learning methods.
Adversary Instantiation: Lower Bounds for Differentially Private Machine Learning
DP formalizes this data leakage through a cryptographic game, where an adversary must predict if a model was trained on a dataset D, or a dataset D' that differs in just one example. If observing the training algorithm does not meaningfully increase the adversary's odds of successfully guessing which dataset the model was trained on, then the algorithm is said to be differentially private.
Extracting Training Data from Large Language Models
We demonstrate our attack on GPT-2, a language model trained on scrapes of the public Internet, and are able to extract hundreds of verbatim text sequences from the model's training data.
Is Private Learning Possible with Instance Encoding?
A private machine learning algorithm hides as much as possible about its training data while still preserving accuracy.
Erratum Concerning the Obfuscated Gradients Attack on Stochastic Activation Pruning
Stochastic Activation Pruning (SAP) (Dhillon et al., 2018) is a defense to adversarial examples that was attacked and found to be broken by the "Obfuscated Gradients" paper (Athalye et al., 2018).
A Partial Break of the Honeypots Defense to Catch Adversarial Attacks
A recent defense proposes to inject "honeypots" into neural networks in order to detect adversarial attacks.
Label-Only Membership Inference Attacks
We empirically show that our label-only membership inference attacks perform on par with prior attacks that required access to model confidences.
Measuring Robustness to Natural Distribution Shifts in Image Classification
We study how robust current ImageNet models are to distribution shifts arising from natural variations in datasets.
Ranked #47 on
Domain Generalization
on VizWiz-Classification
ReMixMatch: Semi-Supervised Learning with Distribution Matching and Augmentation Anchoring
We improve the recently-proposed ``MixMatch semi-supervised learning algorithm by introducing two new techniques: distribution alignment and augmentation anchoring.
Evading Deepfake-Image Detectors with White- and Black-Box Attacks
We show that such forensic classifiers are vulnerable to a range of attacks that reduce the classifier to near-0% accuracy.
Cryptanalytic Extraction of Neural Network Models
We argue that the machine learning problem of model extraction is actually a cryptanalytic problem in disguise, and should be studied as such.
On Adaptive Attacks to Adversarial Example Defenses
Adaptive attacks have (rightfully) become the de facto standard for evaluating defenses to adversarial examples.
Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations
Adversarial examples are malicious inputs crafted to induce misclassification.
FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
Semi-supervised learning (SSL) provides an effective means of leveraging unlabeled data to improve a model's performance.
ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
Distribution alignment encourages the marginal distribution of predictions on unlabeled data to be close to the marginal distribution of ground-truth labels.
Distribution Density, Tails, and Outliers in Machine Learning: Metrics and Applications
We develop techniques to quantify the degree to which a given (training or testing) example is an outlier in the underlying distribution.
When Robustness Doesn’t Promote Robustness: Synthetic vs. Natural Distribution Shifts on ImageNet
We conduct a large experimental comparison of various robustness metrics for image classification.
High Accuracy and High Fidelity Extraction of Neural Networks
In a model extraction attack, an adversary steals a copy of a remotely deployed machine learning model, given oracle prediction access.
Stateful Detection of Black-Box Adversarial Attacks
This is true even when, as is the case in many practical settings, the classifier is hosted as a remote service and so the adversary does not have direct access to the model parameters.
A critique of the DeepSec Platform for Security Analysis of Deep Learning Models
At IEEE S&P 2019, the paper "DeepSec: A Uniform Platform for Security Analysis of Deep Learning Model" aims to to "systematically evaluate the existing adversarial attack and defense methods."
MixMatch: A Holistic Approach to Semi-Supervised Learning
Semi-supervised learning has proven to be a powerful paradigm for leveraging unlabeled data to mitigate the reliance on large labeled datasets.
Prototypical Examples in Deep Learning: Metrics, Characteristics, and Utility
Machine learning (ML) research has investigated prototypes: examples that are representative of the behavior to be learned.
MLSys: The New Frontier of Machine Learning Systems
Machine learning (ML) techniques are enjoying rapidly increasing adoption.
Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness
Excessive invariance is not limited to models trained to be robust to perturbation-based $\ell_p$-norm adversaries.
Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition
Adversarial examples are inputs to machine learning models designed by an adversary to cause an incorrect output.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
On Evaluating Adversarial Robustness
Correctly evaluating defenses against adversarial examples has proven to be extremely difficult.
Unrestricted Adversarial Examples
We introduce a two-player contest for evaluating the safety and robustness of machine learning systems, with a large prize pool.
On the Robustness of the CVPR 2018 White-Box Adversarial Example Defenses
Neural networks are known to be vulnerable to adversarial examples.
The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
This paper describes a testing methodology for quantitatively assessing the risk that rare or unique training-data sequences are unintentionally memorized by generative sequence models---a common type of machine-learning model.
Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
We identify obfuscated gradients, a kind of gradient masking, as a phenomenon that leads to a false sense of security in defenses against adversarial examples.
Audio Adversarial Examples: Targeted Attacks on Speech-to-Text
We construct targeted audio adversarial examples on automatic speech recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Ground-Truth Adversarial Examples
We demonstrate how ground truths can serve to assess the effectiveness of attack techniques, by comparing the adversarial examples produced by those attacks to the ground truths; and also of defense techniques, by computing the distance to the ground truths before and after the defense is applied, and measuring the improvement.
MagNet and "Efficient Defenses Against Adversarial Attacks" are Not Robust to Adversarial Examples
MagNet and "Efficient Defenses..." were recently proposed as a defense to adversarial examples.
Provably Minimally-Distorted Adversarial Examples
Using this approach, we demonstrate that one of the recent ICLR defense proposals, adversarial retraining, provably succeeds at increasing the distortion required to construct adversarial examples by a factor of 4. 2.
Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong
We ask whether a strong defense can be created by combining multiple (possibly weak) defenses.
Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
Neural networks are known to be vulnerable to adversarial examples: inputs that are close to natural inputs but classified incorrectly.
Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
An adversarial example library for constructing attacks, building defenses, and benchmarking both
Towards Evaluating the Robustness of Neural Networks
Defensive distillation is a recently proposed approach that can take an arbitrary neural network, and increase its robustness, reducing the success rate of current attacks' ability to find adversarial examples from $95\%$ to $0. 5\%$.
Defensive Distillation is Not Robust to Adversarial Examples
We show that defensive distillation is not secure: it is no more resistant to targeted misclassification attacks than unprotected neural networks.
