Search Results for author:
Found 35 papers, 21 papers with code
Privacy Backdoors: Stealing Data with Corrupted Pretrained Models
We show that this practice introduces a new risk of privacy backdoors.
Query-Based Adversarial Prompt Generation
Recent work has shown it is possible to construct adversarial examples that cause an aligned language model to emit harmful strings or perform harmful behavior.
Scalable Extraction of Training Data from (Production) Language Models
This paper studies extractable memorization: training data that an adversary can efficiently extract by querying a machine learning model without prior knowledge of the training dataset.
Universal Jailbreak Backdoors from Poisoned Human Feedback
Reinforcement Learning from Human Feedback (RLHF) is used to align large language models to produce helpful and harmless responses.
Privacy Side Channels in Machine Learning Systems
Most current approaches for protecting privacy in machine learning (ML) assume that models exist in a vacuum, when in reality, ML models are part of larger systems that include components for training data filtering, output monitoring, and more.
Evaluating Superhuman Models with Consistency Checks
If machine learning models were to achieve superhuman abilities at various reasoning or decision-making tasks, how would we go about evaluating such models, given that humans would necessarily be poor proxies for ground truth?
Evading Black-box Classifiers Without Breaking Eggs
We then design new attacks that reduce the number of bad queries by $1. 5$-$7. 3\times$, but often at a significant increase in total (non-bad) queries.
Randomness in ML Defenses Helps Persistent Attackers and Hinders Evaluators
It is becoming increasingly imperative to design robust ML defenses.
Poisoning Web-Scale Training Datasets is Practical
Deep learning models are often trained on distributed, webscale datasets crawled from the internet.
Tight Auditing of Differentially Private Machine Learning
Moreover, our auditing scheme requires only two training runs (instead of thousands) to produce tight privacy estimates, by adapting recent advances in tight composition theorems for differential privacy.
Extracting Training Data from Diffusion Models
Image diffusion models such as DALL-E 2, Imagen, and Stable Diffusion have attracted significant attention due to their ability to generate high-quality synthetic images.
Considerations for Differentially Private Learning with Large-Scale Public Pretraining
The performance of differentially private machine learning can be boosted significantly by leveraging the transfer learning capabilities of non-private models pretrained on large public datasets.
Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy
Studying data memorization in neural language models helps us understand the risks (e. g., to privacy or copyright) associated with models regurgitating training data and aids in the development of countermeasures.
Preprocessors Matter! Realistic Decision-Based Attacks on Machine Learning Systems
Decision-based attacks construct adversarial examples against a machine learning (ML) model by making only hard-label queries.
Red-Teaming the Stable Diffusion Safety Filter
We then reverse-engineer the filter and find that while it aims to prevent sexual content, it ignores violence, gore, and other similarly disturbing content.
SNAP: Efficient Extraction of Private Properties with Poisoning
Property inference attacks allow an adversary to extract global properties of the training dataset from a machine learning model.
Measuring Forgetting of Memorized Training Examples
In memorization, models overfit specific training examples and become susceptible to privacy attacks.
Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets
We show that an adversary who can poison a training dataset can cause models trained on this dataset to leak significant private details of training points belonging to other parties.
What Does it Mean for a Language Model to Preserve Privacy?
Language models lack the ability to understand the context and sensitivity of text, and tend to memorize phrases present in their training sets.
Large Language Models Can Be Strong Differentially Private Learners
Differentially Private (DP) learning has seen limited success for building large deep learning models of text, and straightforward attempts at applying Differentially Private Stochastic Gradient Descent (DP-SGD) to NLP tasks have resulted in large performance drops and high computational overhead.
On the Opportunities and Risks of Foundation Models
AI is undergoing a paradigm shift with the rise of models (e. g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.
Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Them
We prove a general hardness reduction between detection and classification of adversarial examples: given a robust detector for attacks at distance {\epsilon} (in some metric), we can build a similarly robust (but inefficient) classifier for attacks at distance {\epsilon}/2.
Data Poisoning Won't Save You From Facial Recognition
We demonstrate that this strategy provides a false sense of security, as it ignores an inherent asymmetry between the parties: users' pictures are perturbed once and for all before being published (at which point they are scraped) and must thereafter fool all future models -- including models trained adaptively against the users' past attacks, or models that use technologies discovered after the attack.
Antipodes of Label Differential Privacy: PATE and ALIBI
We propose two novel approaches based on, respectively, the Laplace mechanism and the PATE framework, and demonstrate their effectiveness on standard benchmarks.
Differentially Private Learning Needs Better Features (or Much More Data)
We demonstrate that differentially private machine learning has not yet reached its "AlexNet moment" on many canonical vision tasks: linear models trained on handcrafted features significantly outperform end-to-end deep neural networks for moderate privacy budgets.
Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations
Adversarial examples are malicious inputs crafted to induce misclassification.
Advances and Open Problems in Federated Learning
FL embodies the principles of focused data collection and minimization, and can mitigate many of the systemic privacy risks and costs resulting from traditional, centralized machine learning and data science approaches.
Adversarial Training and Robustness for Multiple Perturbations
Defenses against adversarial examples, such as adversarial training, are typically tailored to a single perturbation type (e. g., small $\ell_\infty$-noise).
Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness
Excessive invariance is not limited to models trained to be robust to perturbation-based $\ell_p$-norm adversaries.
SentiNet: Detecting Physical Attacks Against Deep Learning Systems
By leveraging the neural network's susceptibility to attacks and by using techniques from model interpretability and object detection as detection mechanisms, SentiNet turns a weakness of a model into a strength.
Cryptography and Security
AdVersarial: Perceptual Ad Blocking meets Adversarial Machine Learning
On the other, we present a concrete set of attacks on visual ad-blockers by constructing adversarial examples in a real web page context.
Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware
As Machine Learning (ML) gets applied to security-critical or sensitive domains, there is a growing need for integrity and privacy for outsourced ML computations.
Ensemble Adversarial Training: Attacks and Defenses
We show that this form of adversarial training converges to a degenerate global minimum, wherein small curvature artifacts near the data points obfuscate a linear approximation of the loss.
The Space of Transferable Adversarial Examples
Adversarial examples are maliciously perturbed inputs designed to mislead machine learning (ML) models at test-time.
Stealing Machine Learning Models via Prediction APIs
In such attacks, an adversary with black-box access, but no prior knowledge of an ML model's parameters or training data, aims to duplicate the functionality of (i. e., "steal") the model.
