Search Results for author:
Found 74 papers, 32 papers with code
On the Difficulty of Defending Self-Supervised Learning against Model Extraction
We construct several novel attacks and find that approaches that train directly on a victim's stolen representations are query efficient and enable high accuracy for downstream models.
Is Fairness Only Metric Deep? Evaluating and Addressing Subgroup Gaps in Deep Metric Learning
Deep metric learning (DML) enables learning with less supervision through its emphasis on the similarity structure of representations.
Bounding Membership Inference
In practice, this means that models need to be trained with DP guarantees that greatly decrease their accuracy.
Differentially Private Speaker Anonymization
We remove speaker information from these attributes by introducing differentially private feature extractors based on an autoencoder and an automatic speech recognizer, respectively, trained using noise layers.
Pipe Overflow: Smashing Voice Authentication for Fun and Profit
Recent years have seen a surge of popularity of acoustics-enabled personal devices powered by machine learning.
Increasing the Cost of Model Extraction with Calibrated Proof of Work
Since we calibrate the effort required to complete the proof-of-work to each query, this only introduces a slight overhead for regular users (up to 2x).
When the Curious Abandon Honesty: Federated Learning Is Not Private
This is because prior efforts exclusively consider passive attackers that are honest-but-curious.
On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning
Machine unlearning, i. e. having a model forget about some of its training data, has become increasingly more important as privacy legislation promotes variants of the right-to-be-forgotten.
Hyperparameter Tuning with Renyi Differential Privacy
For many differentially private algorithms, such as the prominent noisy stochastic gradient descent (DP-SGD), the analysis needed to bound the privacy leakage of a single training run is well understood.
Private Multi-Winner Voting For Machine Learning
We find that our techniques outperform all others in the centralized setting.
A Zest of LIME: Towards Architecture-Independent Model Distances
In this paper, we instead propose to compute distance between black-box models by comparing their Local Interpretable Model-Agnostic Explanations (LIME).
Losing Less: A Loss for Differentially Private Deep Learning
In this paper, we are the first to observe that some of this performance can be recovered when training with a loss tailored to DP-SGD; we challenge cross-entropy as the de facto loss for deep learning with DP.
Context-invariant, multi-variate time series representations
Modern time series corpora, in particular those coming from sensor-based data, exhibit characteristics that have so far not been adequately addressed in the literature on representation learning for time series.
Unrolling SGD: Understanding Factors Influencing Machine Unlearning
In this work, we first taxonomize approaches and metrics of approximate unlearning.
Interpretability in Safety-Critical FinancialTrading Systems
Indeed, we show in our evaluation that errors in the forecasting model's predictions alone are not sufficient for trading decisions made based on these forecasts to yield a negative return.
SoK: Machine Learning Governance
The application of machine learning (ML) in computer systems introduces not only many benefits but also risks to society.
On the Exploitability of Audio Machine Learning Pipelines to Surreptitious Adversarial Examples
In the white-box setting, we instantiate this class with a joint, multi-stage optimization attack.
Bad Characters: Imperceptible NLP Attacks
In this paper, we explore a large class of adversarial examples that can be used to attack text-based models in a black-box setting without making any human-perceptible visual modification to inputs.
Markpainting: Adversarial Machine Learning meets Inpainting
Inpainting is a learned interpolation technique that is based on generative modeling and used to populate masked or missing pieces in an image; it has wide applications in picture editing and retouching.
Dataset Inference: Ownership Resolution in Machine Learning
We thus introduce $dataset$ $inference$, the process of identifying whether a suspected model copy has private knowledge from the original model's dataset, as a defense against model stealing.
Manipulating SGD with Data Ordering Attacks
Machine learning is vulnerable to a wide variety of attacks.
Proof-of-Learning: Definitions and Practice
In particular, our analyses and experiments show that an adversary seeking to illegitimately manufacture a proof-of-learning needs to perform *at least* as much work than is needed for gradient descent itself.
CaPC Learning: Confidential and Private Collaborative Learning
There is currently no method that enables machine learning in such a setting, where both confidentiality and privacy need to be preserved, to prevent both explicit and implicit sharing of data.
Adversary Instantiation: Lower Bounds for Differentially Private Machine Learning
DP formalizes this data leakage through a cryptographic game, where an adversary must predict if a model was trained on a dataset D, or a dataset D' that differs in just one example. If observing the training algorithm does not meaningfully increase the adversary's odds of successfully guessing which dataset the model was trained on, then the algorithm is said to be differentially private.
On the Robustness of Sentiment Analysis for Stock Price Forecasting
Machine learning (ML) models are known to be vulnerable to attacks both at training and test time.
Neighbors From Hell: Voltage Attacks Against Deep Learning Accelerators on Multi-Tenant FPGAs
We find that even when using the strongest attacker circuit, the prediction accuracy of the DL accelerator is not compromised when running at its safe operating frequency.
Data-Free Model Extraction
Current model extraction attacks assume that the adversary has access to a surrogate dataset with characteristics similar to the proprietary data used to train the victim model.
Adversarial Examples in Constrained Domains
To assess how these algorithms perform, we evaluate them in constrained (e. g., network intrusion detection) and unconstrained (e. g., image recognition) domains.
Chasing Your Long Tails: Differentially Private Prediction in Health Care Settings
Our results highlight lesser-known limitations of methods for DP learning in health care, models that exhibit steep tradeoffs between privacy and utility, and models whose predictions are disproportionately influenced by large demographic groups in the training data.
On Attribution of Deepfakes
We discuss the ethical implications of our work, identify where our technique can be used, and highlight that a more meaningful legislative framework is required for a more transparent and ethical use of generative modeling.
Tempered Sigmoid Activations for Deep Learning with Differential Privacy
Because learning sometimes involves sensitive data, machine learning algorithms have been extended to offer privacy for training data.
Label-Only Membership Inference Attacks
We empirically show that our label-only membership inference attacks perform on par with prior attacks that required access to model confidences.
SoK: The Faults in our ASRs: An Overview of Attacks against Automatic Speech Recognition and Speaker Identification Systems
Like other systems based on neural networks, recent research has demonstrated that speech and speaker recognition systems are vulnerable to attacks using manipulated inputs.
Sponge Examples: Energy-Latency Attacks on Neural Networks
The high energy costs of neural network training and inference led to the use of acceleration hardware such as GPUs and TPUs.
SOAR: Second-Order Adversarial Regularization
Adversarial training is a common approach to improving the robustness of deep neural networks against adversarial examples.
On the Robustness of Cooperative Multi-Agent Reinforcement Learning
Our results on the StartCraft II multi-agent benchmark demonstrate that c-MARL teams are highly vulnerable to perturbations applied to one of their agent's observations.
Entangled Watermarks as a Defense against Model Extraction
Such pairs are watermarks, which are not sampled from the task distribution and are only known to the defender.
On the Effectiveness of Mitigating Data Poisoning Attacks with Gradient Shaping
In this work, we study the feasibility of an attack-agnostic defense relying on artifacts that are common to all poisoning attacks.
Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations
Adversarial examples are malicious inputs crafted to induce misclassification.
Machine Unlearning
Once users have shared their data online, it is generally difficult for them to revoke access and ask for the data to be deleted.
Distribution Density, Tails, and Outliers in Machine Learning: Metrics and Applications
We develop techniques to quantify the degree to which a given (training or testing) example is an outlier in the underlying distribution.
Thieves on Sesame Street! Model Extraction of BERT-based APIs
We study the problem of model extraction in natural language processing, in which an adversary with only query access to a victim model attempts to reconstruct a local copy of that model.
Improving Differentially Private Models with Active Learning
Broad adoption of machine learning techniques has increased privacy concerns for models trained on sensitive data such as medical records.
Making the Shoe Fit: Architectures, Initializations, and Tuning for Learning with Privacy
Because learning sometimes involves sensitive data, standard machine-learning algorithms have been extended to offer strong privacy guarantees for training data.
High Accuracy and High Fidelity Extraction of Neural Networks
In a model extraction attack, an adversary steals a copy of a remotely deployed machine learning model, given oracle prediction access.
How Relevant is the Turing Test in the Age of Sophisbots?
Popular culture has contemplated societies of thinking machines for generations, envisioning futures from utopian to dystopian.
Rearchitecting Classification Frameworks For Increased Robustness
and how to design a classification paradigm that leverages these invariances to improve the robustness accuracy trade-off?
MixMatch: A Holistic Approach to Semi-Supervised Learning
Semi-supervised learning has proven to be a powerful paradigm for leveraging unlabeled data to mitigate the reliance on large labeled datasets.
Prototypical Examples in Deep Learning: Metrics, Characteristics, and Utility
Machine learning (ML) research has investigated prototypes: examples that are representative of the behavior to be learned.
Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness
Excessive invariance is not limited to models trained to be robust to perturbation-based $\ell_p$-norm adversaries.
On Evaluating Adversarial Robustness
Correctly evaluating defenses against adversarial examples has proven to be extremely difficult.
Analyzing and Improving Representations with the Soft Nearest Neighbor Loss
We explore and expand the $\textit{Soft Nearest Neighbor Loss}$ to measure the $\textit{entanglement}$ of class manifolds in representation space: i. e., how close pairs of points from the same class are relative to pairs of points from different classes.
A General Approach to Adding Differential Privacy to Iterative Training Procedures
In this work we address the practical challenges of training machine learning models on privacy-sensitive datasets by introducing a modular approach that minimizes changes to training algorithms, provides a variety of configuration strategies for the privacy mechanism, and then isolates and simplifies the critical logic that computes the final privacy guarantees.
A Marauder's Map of Security and Privacy in Machine Learning
We structure our discussion around three of these directions, which we believe are likely to lead to significant progress.
Cryptography and Security
Adversarial Vision Challenge
The NIPS 2018 Adversarial Vision Challenge is a competition to facilitate measurable progress towards robust machine vision models and more generally applicable adversarial attacks.
Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
However, deep learning is often criticized for its lack of robustness in adversarial settings (e. g., vulnerability to adversarial inputs) and general inability to rationalize its predictions.
Adversarial Examples that Fool both Computer Vision and Time-Limited Humans
Machine learning models are vulnerable to adversarial examples: small changes to images can cause computer vision models to make mistakes such as identifying a school bus as an ostrich.
On the Protection of Private Information in Machine Learning Systems: Two Recent Approaches
The recent, remarkable growth of machine learning has led to intense interest in the privacy of the data on which machine learning relies, and to new techniques for preserving privacy.
Ensemble Adversarial Training: Attacks and Defenses
We show that this form of adversarial training converges to a degenerate global minimum, wherein small curvature artifacts near the data points obfuscate a linear approximation of the loss.
Extending Defensive Distillation
Machine learning is vulnerable to adversarial examples: inputs carefully modified to force misclassification.
The Space of Transferable Adversarial Examples
Adversarial examples are maliciously perturbed inputs designed to mislead machine learning (ML) models at test-time.
On the (Statistical) Detection of Adversarial Examples
Specifically, we augment our ML model with an additional output, in which the model is trained to classify all adversarial inputs.
Adversarial Attacks on Neural Network Policies
Machine learning classifiers are known to be vulnerable to inputs maliciously constructed by adversaries to force misclassification.
Towards the Science of Security and Privacy in Machine Learning
Advances in machine learning (ML) in recent years have enabled a dizzying array of applications such as data analytics, autonomous systems, and security diagnostics.
Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data
The approach combines, in a black-box fashion, multiple models trained with disjoint datasets, such as records from different subsets of users.
Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
An adversarial example library for constructing attacks, building defenses, and benchmarking both
Adversarial Perturbations Against Deep Neural Networks for Malware Classification
Deep neural networks, like many other machine learning models, have recently been shown to lack robustness against adversarially crafted inputs.
Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
We demonstrate our attacks on two commercial machine learning classification systems from Amazon (96. 19% misclassification rate) and Google (88. 94%) using only 800 queries of the victim model, thereby showing that existing machine learning approaches are in general vulnerable to systematic black-box attacks regardless of their structure.
Crafting Adversarial Input Sequences for Recurrent Neural Networks
Machine learning models are frequently used to solve complex security problems, as well as to make decisions in sensitive situations like guiding autonomous vehicles or predicting financial market behaviors.
Detection under Privileged Information
In this paper, we consider an alternate learning approach that trains models using "privileged" information--features available at training time but not at runtime--to improve the accuracy and resilience of detection systems.
Practical Black-Box Attacks against Machine Learning
Our attack strategy consists in training a local model to substitute for the target DNN, using inputs synthetically generated by an adversary and labeled by the target DNN.
The Limitations of Deep Learning in Adversarial Settings
In this work, we formalize the space of adversaries against deep neural networks (DNNs) and introduce a novel class of algorithms to craft adversarial samples based on a precise understanding of the mapping between inputs and outputs of DNNs.
Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks
In this work, we introduce a defensive mechanism called defensive distillation to reduce the effectiveness of adversarial samples on DNNs.
