Search Results for author:
Found 79 papers, 41 papers with code
Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
An adversarial example library for constructing attacks, building defenses, and benchmarking both
Indicators of Attack Failure: Debugging and Improving Optimization of Adversarial Examples
Evaluating robustness of machine-learning models to adversarial examples is a challenging problem.
Universal and Transferable Adversarial Attacks on Aligned Language Models
Specifically, our approach finds a suffix that, when attached to a wide range of queries for an LLM to produce objectionable content, aims to maximize the probability that the model produces an affirmative response (rather than refusing to answer).
Reverse-Engineering Decoding Strategies Given Blackbox Access to a Language Generation System
Neural language models are increasingly deployed into APIs and websites that allow a user to pass in a prompt and receive generated text.
MixMatch: A Holistic Approach to Semi-Supervised Learning
Semi-supervised learning has proven to be a powerful paradigm for leveraging unlabeled data to mitigate the reliance on large labeled datasets.
ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
Distribution alignment encourages the marginal distribution of predictions on unlabeled data to be close to the marginal distribution of ground-truth labels.
FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
Semi-supervised learning (SSL) provides an effective means of leveraging unlabeled data to improve a model's performance.
Deduplicating Training Data Makes Language Models Better
As a result, over 1% of the unprompted output of language models trained on these datasets is copied verbatim from the training data.
Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
We identify obfuscated gradients, a kind of gradient masking, as a phenomenon that leads to a false sense of security in defenses against adversarial examples.
Towards Evaluating the Robustness of Neural Networks
Defensive distillation is a recently proposed approach that can take an arbitrary neural network, and increase its robustness, reducing the success rate of current attacks' ability to find adversarial examples from $95\%$ to $0. 5\%$.
Membership Inference Attacks From First Principles
A membership inference attack allows an adversary to query a trained machine learning model to predict whether or not a particular example was contained in the model's training dataset.
On Evaluating Adversarial Robustness
Correctly evaluating defenses against adversarial examples has proven to be extremely difficult.
Unrestricted Adversarial Examples
We introduce a two-player contest for evaluating the safety and robustness of machine learning systems, with a large prize pool.
Audio Adversarial Examples: Targeted Attacks on Speech-to-Text
We construct targeted audio adversarial examples on automatic speech recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Quantifying Memorization Across Neural Language Models
Large language models (LMs) have been shown to memorize parts of their training data, and when prompted appropriately, they will emit the memorized training data verbatim.
Extracting Training Data from Large Language Models
We demonstrate our attack on GPT-2, a language model trained on scrapes of the public Internet, and are able to extract hundreds of verbatim text sequences from the model's training data.
ReMixMatch: Semi-Supervised Learning with Distribution Matching and Augmentation Anchoring
We improve the recently-proposed ``MixMatch semi-supervised learning algorithm by introducing two new techniques: distribution alignment and augmentation anchoring.
Measuring Robustness to Natural Distribution Shifts in Image Classification
We study how robust current ImageNet models are to distribution shifts arising from natural variations in datasets.
Ranked #41 on
Domain Generalization
on VizWiz-Classification
On Adaptive Attacks to Adversarial Example Defenses
Adaptive attacks have (rightfully) become the de facto standard for evaluating defenses to adversarial examples.
Label-Only Membership Inference Attacks
We empirically show that our label-only membership inference attacks perform on par with prior attacks that required access to model confidences.
AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation
We extend semi-supervised learning to the problem of domain adaptation to learn significantly higher-accuracy models that train on one data distribution and test on a different one.
Semi-supervised Domain Adaptation
Unsupervised Domain Adaptation
Provably Minimally-Distorted Adversarial Examples
Using this approach, we demonstrate that one of the recent ICLR defense proposals, adversarial retraining, provably succeeds at increasing the distortion required to construct adversarial examples by a factor of 4. 2.
Cryptanalytic Extraction of Neural Network Models
We argue that the machine learning problem of model extraction is actually a cryptanalytic problem in disguise, and should be studied as such.
(Certified!!) Adversarial Robustness for Free!
In this paper we show how to achieve state-of-the-art certified adversarial robustness to 2-norm bounded perturbations by relying exclusively on off-the-shelf pretrained models.
Poisoning and Backdooring Contrastive Learning
Multimodal contrastive learning methods like CLIP train on noisy and uncurated training datasets.
Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations
Adversarial examples are malicious inputs crafted to induce misclassification.
Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition
Adversarial examples are inputs to machine learning models designed by an adversary to cause an incorrect output.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Considerations for Differentially Private Learning with Large-Scale Public Pretraining
The performance of differentially private machine learning can be boosted significantly by leveraging the transfer learning capabilities of non-private models pretrained on large public datasets.
Evading Adversarial Example Detection Defenses with Orthogonal Projected Gradient Descent
Evading adversarial example detection defenses requires finding adversarial examples that must simultaneously (a) be misclassified by the model and (b) be detected as non-adversarial.
Part-Based Models Improve Adversarial Robustness
We show that combining human prior knowledge with end-to-end learning can improve the robustness of deep neural networks by introducing a part-based model for object classification.
Preprocessors Matter! Realistic Decision-Based Attacks on Machine Learning Systems
Decision-based attacks construct adversarial examples against a machine learning (ML) model by making only hard-label queries.
Evading Black-box Classifiers Without Breaking Eggs
We then design new attacks that reduce the number of bad queries by $1. 5$-$7. 3\times$, but often at a significant increase in total (non-bad) queries.
Stateful Detection of Black-Box Adversarial Attacks
This is true even when, as is the case in many practical settings, the classifier is hosted as a remote service and so the adversary does not have direct access to the model parameters.
Is Private Learning Possible with Instance Encoding?
A private machine learning algorithm hides as much as possible about its training data while still preserving accuracy.
Defensive Distillation is Not Robust to Adversarial Examples
We show that defensive distillation is not secure: it is no more resistant to targeted misclassification attacks than unprotected neural networks.
Data Poisoning Won't Save You From Facial Recognition
We demonstrate that this strategy provides a false sense of security, as it ignores an inherent asymmetry between the parties: users' pictures are perturbed once and for all before being published (at which point they are scraped) and must thereafter fool all future models -- including models trained adaptively against the users' past attacks, or models that use technologies discovered after the attack.
MagNet and "Efficient Defenses Against Adversarial Attacks" are Not Robust to Adversarial Examples
MagNet and "Efficient Defenses..." were recently proposed as a defense to adversarial examples.
On the Robustness of the CVPR 2018 White-Box Adversarial Example Defenses
Neural networks are known to be vulnerable to adversarial examples.
Initialization Matters for Adversarial Transfer Learning
Based on this, we propose Robust Linear Initialization (RoLI) for adversarial finetuning, which initializes the linear head with the weights obtained by adversarial linear probing to maximally inherit the robustness from pretraining.
The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
This paper describes a testing methodology for quantitatively assessing the risk that rare or unique training-data sequences are unintentionally memorized by generative sequence models---a common type of machine-learning model.
Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
Neural networks are known to be vulnerable to adversarial examples: inputs that are close to natural inputs but classified incorrectly.
Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong
We ask whether a strong defense can be created by combining multiple (possibly weak) defenses.
Prototypical Examples in Deep Learning: Metrics, Characteristics, and Utility
Machine learning (ML) research has investigated prototypes: examples that are representative of the behavior to be learned.
Ground-Truth Adversarial Examples
We demonstrate how ground truths can serve to assess the effectiveness of attack techniques, by comparing the adversarial examples produced by those attacks to the ground truths; and also of defense techniques, by computing the distance to the ground truths before and after the defense is applied, and measuring the improvement.
Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness
Excessive invariance is not limited to models trained to be robust to perturbation-based $\ell_p$-norm adversaries.
MLSys: The New Frontier of Machine Learning Systems
Machine learning (ML) techniques are enjoying rapidly increasing adoption.
A critique of the DeepSec Platform for Security Analysis of Deep Learning Models
At IEEE S&P 2019, the paper "DeepSec: A Uniform Platform for Security Analysis of Deep Learning Model" aims to to "systematically evaluate the existing adversarial attack and defense methods."
High Accuracy and High Fidelity Extraction of Neural Networks
In a model extraction attack, an adversary steals a copy of a remotely deployed machine learning model, given oracle prediction access.
Distribution Density, Tails, and Outliers in Machine Learning: Metrics and Applications
We develop techniques to quantify the degree to which a given (training or testing) example is an outlier in the underlying distribution.
Evading Deepfake-Image Detectors with White- and Black-Box Attacks
We show that such forensic classifiers are vulnerable to a range of attacks that reduce the classifier to near-0% accuracy.
A Partial Break of the Honeypots Defense to Catch Adversarial Attacks
A recent defense proposes to inject "honeypots" into neural networks in order to detect adversarial attacks.
Erratum Concerning the Obfuscated Gradients Attack on Stochastic Activation Pruning
Stochastic Activation Pruning (SAP) (Dhillon et al., 2018) is a defense to adversarial examples that was attacked and found to be broken by the "Obfuscated Gradients" paper (Athalye et al., 2018).
Adversary Instantiation: Lower Bounds for Differentially Private Machine Learning
DP formalizes this data leakage through a cryptographic game, where an adversary must predict if a model was trained on a dataset D, or a dataset D' that differs in just one example. If observing the training algorithm does not meaningfully increase the adversary's odds of successfully guessing which dataset the model was trained on, then the algorithm is said to be differentially private.
Poisoning the Unlabeled Dataset of Semi-Supervised Learning
Our attacks are highly effective across datasets and semi-supervised learning methods.
Handcrafted Backdoors in Deep Neural Networks
When machine learning training is outsourced to third parties, $backdoor$ $attacks$ become practical as the third party who trains the model may act maliciously to inject hidden behaviors into the otherwise accurate model.
Unsolved Problems in ML Safety
Machine learning (ML) systems are rapidly increasing in size, are acquiring new capabilities, and are increasingly deployed in high-stakes settings.
When Robustness Doesn’t Promote Robustness: Synthetic vs. Natural Distribution Shifts on ImageNet
We conduct a large experimental comparison of various robustness metrics for image classification.
Debugging Differential Privacy: A Case Study for Privacy Auditing
Differential Privacy can provide provable privacy guarantees for training data in machine learning.
Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets
We show that an adversary who can poison a training dataset can cause models trained on this dataset to leak significant private details of training points belonging to other parties.
The Privacy Onion Effect: Memorization is Relative
Machine learning models trained on private datasets have been shown to leak their private data.
Increasing Confidence in Adversarial Robustness Evaluations
Hundreds of defenses have been proposed to make deep neural networks robust against minimal (adversarial) input perturbations.
Measuring Forgetting of Memorized Training Examples
In memorization, models overfit specific training examples and become susceptible to privacy attacks.
No Free Lunch in "Privacy for Free: How does Dataset Condensation Help Privacy"
New methods designed to preserve data privacy require careful scrutiny.
Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy
Studying data memorization in neural language models helps us understand the risks (e. g., to privacy or copyright) associated with models regurgitating training data and aids in the development of countermeasures.
Publishing Efficient On-device Models Increases Adversarial Vulnerability
We then show that the vulnerability increases as the similarity between a full-scale and its efficient model increase.
Extracting Training Data from Diffusion Models
Image diffusion models such as DALL-E 2, Imagen, and Stable Diffusion have attracted significant attention due to their ability to generate high-quality synthetic images.
Tight Auditing of Differentially Private Machine Learning
Moreover, our auditing scheme requires only two training runs (instead of thousands) to produce tight privacy estimates, by adapting recent advances in tight composition theorems for differential privacy.
Poisoning Web-Scale Training Datasets is Practical
Deep learning models are often trained on distributed, webscale datasets crawled from the internet.
Randomness in ML Defenses Helps Persistent Attackers and Hinders Evaluators
It is becoming increasingly imperative to design robust ML defenses.
Effective Prompt Extraction from Language Models
In experiments with 3 different sources of prompts and 11 underlying large language models, we find that simple text-based attacks can in fact reveal prompts with high probability.
A LLM Assisted Exploitation of AI-Guardian
Large language models (LLMs) are now highly capable at a diverse range of tasks.
Identifying and Mitigating the Security Risks of Generative AI
However, GenAI can be used just as well by attackers to generate new attacks and increase the velocity and efficacy of existing attacks.
Privacy Side Channels in Machine Learning Systems
Most current approaches for protecting privacy in machine learning (ML) assume that models exist in a vacuum, when in reality, ML models are part of larger systems that include components for training data filtering, output monitoring, and more.
Scalable Extraction of Training Data from (Production) Language Models
This paper studies extractable memorization: training data that an adversary can efficiently extract by querying a machine learning model without prior knowledge of the training dataset.
Query-Based Adversarial Prompt Generation
Recent work has shown it is possible to construct adversarial examples that cause an aligned language model to emit harmful strings or perform harmful behavior.
Diffusion Denoising as a Certified Defense against Clean-label Poisoning
We present a certified defense to clean-label poisoning attacks.
Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models
In this paper, we unveil a new vulnerability: the privacy backdoor attack.
Forcing Diffuse Distributions out of Language Models
Despite being trained specifically to follow user instructions, today's language models perform poorly when instructed to produce random outputs.
