Search Results for author: Somesh Jha

Found 80 papers, 32 papers with code

Theoretically Principled Trade-off for Stateful Defenses against Query-Based Black-Box Attacks

no code implementations30 Jul 2023 Ashish Hooda, Neal Mangaokar, Ryan Feng, Kassem Fawaz, Somesh Jha, Atul Prakash

This work aims to address this gap by offering a theoretical characterization of the trade-off between detection and false positive rates for stateful defenses.

Pareto-Secure Machine Learning (PSML): Fingerprinting and Securing Inference Serving Systems

no code implementations3 Jul 2023 Debopam Sanyal, Jui-Tse Hung, Manav Agrawal, Prahlad Jasti, Shahab Nikkhoo, Somesh Jha, Tianhao Wang, Sibin Mohan, Alexey Tumanov

Second, we counter the proposed attack with a noise-based defense mechanism that thwarts fingerprinting by adding noise to the specified performance metrics.

Model extraction

Two Heads are Better than One: Towards Better Adversarial Robustness by Combining Transduction and Rejection

no code implementations27 May 2023 Nils Palumbo, Yang Guo, Xi Wu, Jiefeng Chen, YIngyu Liang, Somesh Jha

Nevertheless, under recent strong adversarial attacks (GMSA, which has been shown to be much more effective than AutoAttack against transduction), Goldwasser et al.'s work was shown to have low performance in a practical deep-learning setting.

Adversarial Robustness

Rethink Diversity in Deep Learning Testing

no code implementations25 May 2023 Zi Wang, Jihye Choi, Somesh Jha

Deep neural networks (DNNs) have demonstrated extraordinary capabilities and are an integral part of modern software systems.

DNN Testing

Stratified Adversarial Robustness with Rejection

1 code implementation2 May 2023 Jiefeng Chen, Jayaram Raghuram, Jihye Choi, Xi Wu, YIngyu Liang, Somesh Jha

We theoretically analyze the stratified rejection setting and propose a novel defense method -- Adversarial Training with Consistent Prediction-based Rejection (CPR) -- for building a robust selective classifier.

Adversarial Robustness Robust classification

ASPEST: Bridging the Gap Between Active Learning and Selective Prediction

1 code implementation7 Apr 2023 Jiefeng Chen, Jinsung Yoon, Sayna Ebrahimi, Sercan Arik, Somesh Jha, Tomas Pfister

In this work, we bridge selective prediction and active learning, proposing a new learning paradigm called active selective prediction which learns to query more informative samples from the shifted target domain while increasing accuracy and coverage.

Active Learning

Efficient Symbolic Reasoning for Neural-Network Verification

no code implementations23 Mar 2023 Zi Wang, Somesh Jha, Krishnamurthy, Dvijotham

They allow us to encode many verification problems for neural networks as quadratic programs.

Stateful Defenses for Machine Learning Models Are Not Yet Secure Against Black-box Attacks

1 code implementation11 Mar 2023 Ryan Feng, Ashish Hooda, Neal Mangaokar, Kassem Fawaz, Somesh Jha, Atul Prakash

Such stateful defenses aim to defend against black-box attacks by tracking the query history and detecting and rejecting queries that are "similar" and thus preventing black-box attacks from finding useful gradients and making progress towards finding adversarial attacks within a reasonable query budget.

The Trade-off between Universality and Label Efficiency of Representations from Contrastive Learning

1 code implementation28 Feb 2023 Zhenmei Shi, Jiefeng Chen, Kunyang Li, Jayaram Raghuram, Xi Wu, YIngyu Liang, Somesh Jha

foundation models) has recently become a prevalent learning paradigm, where one first pre-trains a representation using large-scale unlabeled data, and then learns simple predictors on top of the representation using small labeled data from the downstream tasks.

Contrastive Learning

Learning Modulo Theories

no code implementations26 Jan 2023 Matt Fredrikson, Kaiji Lu, Saranya Vijayakumar, Somesh Jha, Vijay Ganesh, Zifan Wang

Recent techniques that integrate \emph{solver layers} into Deep Neural Networks (DNNs) have shown promise in bridging a long-standing gap between inductive learning and symbolic reasoning techniques.

Federated Boosted Decision Trees with Differential Privacy

1 code implementation6 Oct 2022 Samuel Maddock, Graham Cormode, Tianhao Wang, Carsten Maple, Somesh Jha

There is great demand for scalable, secure, and efficient privacy-preserving machine learning models that can be trained over distributed data.

Privacy Preserving

Overparameterization from Computational Constraints

no code implementations27 Aug 2022 Sanjam Garg, Somesh Jha, Saeed Mahloujifar, Mohammad Mahmoody, Mingyuan Wang

In particular, for computationally bounded learners, we extend the recent result of Bubeck and Sellke [NeurIPS'2021] which shows that robust models might need more parameters, to the computational regime and show that bounded learners could provably need an even larger number of parameters.

Constraining the Attack Space of Machine Learning Models with Distribution Clamping Preprocessing

no code implementations18 May 2022 Ryan Feng, Somesh Jha, Atul Prakash

Preprocessing and outlier detection techniques have both been applied to neural networks to increase robustness with varying degrees of success.

BIG-bench Machine Learning object-detection +2

Optimal Membership Inference Bounds for Adaptive Composition of Sampled Gaussian Mechanisms

no code implementations12 Apr 2022 Saeed Mahloujifar, Alexandre Sablayrolles, Graham Cormode, Somesh Jha

A common countermeasure against MI attacks is to utilize differential privacy (DP) during model training to mask the presence of individual examples.

Concept-based Explanations for Out-Of-Distribution Detectors

1 code implementation4 Mar 2022 Jihye Choi, Jayaram Raghuram, Ryan Feng, Jiefeng Chen, Somesh Jha, Atul Prakash

Based on these metrics, we propose an unsupervised framework for learning a set of concepts that satisfy the desired properties of high detection completeness and concept separability, and demonstrate its effectiveness in providing concept-based explanations for diverse off-the-shelf OOD detectors.

Out of Distribution (OOD) Detection

A Quantitative Geometric Approach to Neural-Network Smoothness

1 code implementation2 Mar 2022 Zi Wang, Gautam Prakriya, Somesh Jha

In this work, we provide a unified theoretical framework, a quantitative geometric approach, to address the Lipschitz constant estimation.

D4: Detection of Adversarial Diffusion Deepfakes Using Disjoint Ensembles

no code implementations11 Feb 2022 Ashish Hooda, Neal Mangaokar, Ryan Feng, Kassem Fawaz, Somesh Jha, Atul Prakash

D4 uses an ensemble of models over disjoint subsets of the frequency spectrum to significantly improve adversarial robustness.

Adversarial Robustness DeepFake Detection +1

An Exploration of Multicalibration Uniform Convergence Bounds

no code implementations9 Feb 2022 Harrison Rosenberg, Robi Bhattacharjee, Kassem Fawaz, Somesh Jha

Given the prevalence of ERM sample complexity bounds, our proposed framework enables machine learning practitioners to easily understand the convergence behavior of multicalibration error for a myriad of classifier architectures.

BIG-bench Machine Learning Fairness

Revisiting Adversarial Robustness of Classifiers With a Reject Option

no code implementations AAAI Workshop AdvML 2022 Jiefeng Chen, Jayaram Raghuram, Jihye Choi, Xi Wu, YIngyu Liang, Somesh Jha

Motivated by this metric, we propose novel loss functions and a robust training method -- \textit{stratified adversarial training with rejection} (SATR) -- for a classifier with reject option, where the goal is to accept and correctly-classify small input perturbations, while allowing the rejection of larger input perturbations that cannot be correctly classified.

Adversarial Robustness Image Classification

Towards Evaluating the Robustness of Neural Networks Learned by Transduction

1 code implementation ICLR 2022 Jiefeng Chen, Xi Wu, Yang Guo, YIngyu Liang, Somesh Jha

There has been emerging interest in using transductive learning for adversarial robustness (Goldwasser et al., NeurIPS 2020; Wu et al., ICML 2020; Wang et al., ArXiv 2021).

Adversarial Robustness Bilevel Optimization +1

Less is More: Dimension Reduction Finds On-Manifold Adversarial Examples in Hard-Label Attacks

no code implementations29 Sep 2021 Washington Garcia, Pin-Yu Chen, Somesh Jha, Hamilton Scott Clouse, Kevin R. B. Butler

It was recently shown in the gradient-level setting that regular adversarial examples leave the data manifold, while their on-manifold counterparts are in fact generalization errors.

Dimensionality Reduction Image Classification

Privacy Implications of Shuffling

no code implementations ICLR 2022 Casey Meehan, Amrita Roy Chowdhury, Kamalika Chaudhuri, Somesh Jha

\ldp deployments are vulnerable to inference attacks as an adversary can link the noisy responses to their identity and subsequently, auxiliary information using the \textit{order} of the data.

Fast and Sample-Efficient Domain Adaptation for Autoencoder-Based End-to-End Communication

no code implementations29 Sep 2021 Jayaram Raghuram, Yijing Zeng, Dolores Garcia, Somesh Jha, Suman Banerjee, Joerg Widmer, Rafael Ruiz

In this paper, we address the setting where the target domain has only limited labeled data from a distribution that is expected to change frequently.

Domain Adaptation

Fairness Properties of Face Recognition and Obfuscation Systems

1 code implementation5 Aug 2021 Harrison Rosenberg, Brian Tang, Kassem Fawaz, Somesh Jha

We answer this question with an analytical and empirical exploration of recent face obfuscation systems.

Face Recognition Fairness

Few-Shot Domain Adaptation For End-to-End Communication

1 code implementation2 Aug 2021 Jayaram Raghuram, Yijing Zeng, Dolores García Martí, Rafael Ruiz Ortiz, Somesh Jha, Joerg Widmer, Suman Banerjee

The problem of end-to-end learning of a communication system using an autoencoder -- consisting of an encoder, channel, and decoder modeled using neural networks -- has recently been shown to be an effective approach.

Domain Adaptation Semi-supervised Domain Adaptation

Detecting Errors and Estimating Accuracy on Unlabeled Data with Self-training Ensembles

1 code implementation NeurIPS 2021 Jiefeng Chen, Frederick Liu, Besim Avci, Xi Wu, YIngyu Liang, Somesh Jha

This observation leads to two challenging tasks: (1) unsupervised accuracy estimation, which aims to estimate the accuracy of a pre-trained classifier on a set of unlabeled test inputs; (2) error detection, which aims to identify mis-classified test inputs.

Towards Adversarial Robustness via Transductive Learning

no code implementations15 Jun 2021 Jiefeng Chen, Yang Guo, Xi Wu, Tianqi Li, Qicheng Lao, YIngyu Liang, Somesh Jha

Compared to traditional "test-time" defenses, these defense mechanisms "dynamically retrain" the model based on test time input via transductive learning; and theoretically, attacking these defenses boils down to bilevel optimization, which seems to raise the difficulty for adaptive attacks.

Adversarial Robustness Bilevel Optimization +1

A Shuffling Framework for Local Differential Privacy

no code implementations11 Jun 2021 Casey Meehan, Amrita Roy Chowdhury, Kamalika Chaudhuri, Somesh Jha

ldp deployments are vulnerable to inference attacks as an adversary can link the noisy responses to their identity and subsequently, auxiliary information using the order of the data.

Hard-label Manifolds: Unexpected Advantages of Query Efficiency for Finding On-manifold Adversarial Examples

no code implementations4 Mar 2021 Washington Garcia, Pin-Yu Chen, Somesh Jha, Scott Clouse, Kevin R. B. Butler

It was recently shown in the gradient-level setting that regular adversarial examples leave the data manifold, while their on-manifold counterparts are in fact generalization errors.

Dimensionality Reduction Image Classification

Exploring Adversarial Robustness of Deep Metric Learning

1 code implementation14 Feb 2021 Thomas Kobber Panum, Zi Wang, Pengyu Kan, Earlence Fernandes, Somesh Jha

Deep Metric Learning (DML), a widely-used technique, involves learning a distance metric between pairs of samples.

Adversarial Robustness Metric Learning

CaPC Learning: Confidential and Private Collaborative Learning

1 code implementation ICLR 2021 Christopher A. Choquette-Choo, Natalie Dullerud, Adam Dziedzic, Yunxiang Zhang, Somesh Jha, Nicolas Papernot, Xiao Wang

There is currently no method that enables machine learning in such a setting, where both confidentiality and privacy need to be preserved, to prevent both explicit and implicit sharing of data.

Fairness Federated Learning

Test-Time Adaptation and Adversarial Robustness

no code implementations1 Jan 2021 Xi Wu, Yang Guo, Tianqi Li, Jiefeng Chen, Qicheng Lao, YIngyu Liang, Somesh Jha

On the positive side, we show that, if one is allowed to access the training data, then Domain Adversarial Neural Networks (${\sf DANN}$), an algorithm designed for unsupervised domain adaptation, can provide nontrivial robustness in the test-time maximin threat model against strong transfer attacks and adaptive fixed point attacks.

Adversarial Robustness Unsupervised Domain Adaptation

Generalized Universal Approximation for Certified Networks

no code implementations1 Jan 2021 Zi Wang, Aws Albarghouthi, Somesh Jha

To certify safety and robustness of neural networks, researchers have successfully applied abstract interpretation, primarily using interval bound propagation.

Adversarial Deep Metric Learning

no code implementations1 Jan 2021 Thomas Kobber Panum, Zi Wang, Pengyu Kan, Earlence Fernandes, Somesh Jha

To the best of our knowledge, we are the first to systematically analyze this dependence effect and propose a principled approach for robust training of deep metric learning networks that accounts for the nuances of metric losses.

Metric Learning

Sample Complexity of Adversarially Robust Linear Classification on Separated Data

no code implementations19 Dec 2020 Robi Bhattacharjee, Somesh Jha, Kamalika Chaudhuri

This shows that for very well-separated data, convergence rates of $O(\frac{1}{n})$ are achievable, which is not the case otherwise.

Adversarial Robustness Classification +1

ShadowNet: A Secure and Efficient On-device Model Inference System for Convolutional Neural Networks

no code implementations11 Nov 2020 Zhichuang Sun, Ruimin Sun, Changming Liu, Amrita Roy Chowdhury, Long Lu, Somesh Jha

ShadowNet protects the model privacy with Trusted Execution Environment (TEE) while securely outsourcing the heavy linear layers of the model to the untrusted hardware accelerators.

Informative Outlier Matters: Robustifying Out-of-distribution Detection Using Outlier Mining

no code implementations28 Sep 2020 Jiefeng Chen, Yixuan Li, Xi Wu, YIngyu Liang, Somesh Jha

We show that, by mining informative auxiliary OOD data, one can significantly improve OOD detection performance, and somewhat surprisingly, generalize to unseen adversarial attacks.

Out-of-Distribution Detection Out of Distribution (OOD) Detection

A General Framework For Detecting Anomalous Inputs to DNN Classifiers

1 code implementation29 Jul 2020 Jayaram Raghuram, Varun Chandrasekaran, Somesh Jha, Suman Banerjee

We propose an unsupervised anomaly detection framework based on the internal DNN layer representations in the form of a meta-algorithm with configurable components.

Image Classification Unsupervised Anomaly Detection

Interval Universal Approximation for Neural Networks

no code implementations12 Jul 2020 Zi Wang, Aws Albarghouthi, Gautam Prakriya, Somesh Jha

This is a crucial question, as our constructive proof of IUA is exponential in the size of the approximation domain.

Robust and Accurate Authorship Attribution via Program Normalization

no code implementations1 Jul 2020 Yizhen Wang, Mohannad Alhanahnah, Ke Wang, Mihai Christodorescu, Somesh Jha

To address these emerging issues, we formulate this security challenge into a general threat model, the $\textit{relational adversary}$, that allows an arbitrary number of the semantics-preserving transformations to be applied to an input in any problem space.

Image Classification Malware Detection

ATOM: Robustifying Out-of-distribution Detection Using Outlier Mining

1 code implementation26 Jun 2020 Jiefeng Chen, Yixuan Li, Xi Wu, YIngyu Liang, Somesh Jha

We show that, by mining informative auxiliary OOD data, one can significantly improve OOD detection performance, and somewhat surprisingly, generalize to unseen adversarial attacks.

Out-of-Distribution Detection Out of Distribution (OOD) Detection

Representation Bayesian Risk Decompositions and Multi-Source Domain Adaptation

no code implementations22 Apr 2020 Xi Wu, Yang Guo, Jiefeng Chen, YIngyu Liang, Somesh Jha, Prasad Chalasani

Recent studies provide hints and failure examples for domain invariant representation learning, a common approach for this problem, but the explanations provided are somewhat different and do not provide a unified picture.

Domain Adaptation Representation Learning

Robust Out-of-distribution Detection for Neural Networks

1 code implementation AAAI Workshop AdvML 2022 Jiefeng Chen, Yixuan Li, Xi Wu, YIngyu Liang, Somesh Jha

Formally, we extensively study the problem of Robust Out-of-Distribution Detection on common OOD detection approaches, and show that state-of-the-art OOD detectors can be easily fooled by adding small perturbations to the in-distribution and OOD inputs.

Out-of-Distribution Detection Out of Distribution (OOD) Detection

Face-Off: Adversarial Face Obfuscation

1 code implementation19 Mar 2020 Chuhan Gao, Varun Chandrasekaran, Kassem Fawaz, Somesh Jha

We implement and evaluate Face-Off to find that it deceives three commercial face recognition services from Microsoft, Amazon, and Face++.

Cryptography and Security

Analyzing Accuracy Loss in Randomized Smoothing Defenses

no code implementations3 Mar 2020 Yue Gao, Harrison Rosenberg, Kassem Fawaz, Somesh Jha, Justin Hsu

In test-time attacks an adversary crafts adversarial examples, which are specially crafted perturbations imperceptible to humans which, when added to an input example, force a machine learning model to misclassify the given input example.

Autonomous Driving BIG-bench Machine Learning +2

CAUSE: Learning Granger Causality from Event Sequences using Attribution Methods

1 code implementation ICML 2020 Wei Zhang, Thomas Kobber Panum, Somesh Jha, Prasad Chalasani, David Page

We study the problem of learning Granger causality between event types from asynchronous, interdependent, multi-type event sequences.

GRAPHITE: Generating Automatic Physical Examples for Machine-Learning Attacks on Computer Vision Systems

1 code implementation17 Feb 2020 Ryan Feng, Neal Mangaokar, Jiefeng Chen, Earlence Fernandes, Somesh Jha, Atul Prakash

We address three key requirements for practical attacks for the real-world: 1) automatically constraining the size and shape of the attack so it can be applied with stickers, 2) transform-robustness, i. e., robustness of a attack to environmental physical variations such as viewpoint and lighting changes, and 3) supporting attacks in not only white-box, but also black-box hard-label scenarios, so that the adversary can attack proprietary models.

BIG-bench Machine Learning General Classification +1

Semantic Robustness of Models of Source Code

1 code implementation7 Feb 2020 Goutham Ramakrishnan, Jordan Henkel, Zi Wang, Aws Albarghouthi, Somesh Jha, Thomas Reps

Deep neural networks are vulnerable to adversarial examples - small input perturbations that result in incorrect predictions.

Generating Semantic Adversarial Examples with Differentiable Rendering

no code implementations2 Oct 2019 Lakshya Jain, Wilson Wu, Steven Chen, Uyeong Jang, Varun Chandrasekaran, Sanjit Seshia, Somesh Jha

In this paper we explore semantic adversarial examples (SAEs) where an attacker creates perturbations in the semantic space representing the environment that produces input for the ML model.

Autonomous Driving

On the Need for Topology-Aware Generative Models for Manifold-Based Defenses

no code implementations ICLR 2020 Uyeong Jang, Susmit Jha, Somesh Jha

These defenses rely on the assumption that data lie in a manifold of a lower dimension than the input space.

Data Augmentation

Improving Utility and Security of the Shuffler-based Differential Privacy

1 code implementation30 Aug 2019 Tianhao Wang, Bolin Ding, Min Xu, Zhicong Huang, Cheng Hong, Jingren Zhou, Ninghui Li, Somesh Jha

When collecting information, local differential privacy (LDP) alleviates privacy concerns of users because their private information is randomized before being sent it to the central aggregator.

Data-Dependent Differentially Private Parameter Learning for Directed Graphical Models

no code implementations ICML 2020 Amrita Roy Chowdhury, Theodoros Rekatsinas, Somesh Jha

Our solution optimizes for the utility of inference queries over the DGM and \textit{adds noise that is customized to the properties of the private input dataset and the graph structure of the DGM}.

Adversarially Robust Learning Could Leverage Computational Hardness

no code implementations28 May 2019 Sanjam Garg, Somesh Jha, Saeed Mahloujifar, Mohammad Mahmoody

On the reverse directions, we also show that the existence of such learning task in which computational robustness beats information theoretic robustness requires computational hardness by implying (average-case) hardness of NP.

An Investigation of Data Poisoning Defenses for Online Learning

no code implementations28 May 2019 Yizhen Wang, Somesh Jha, Kamalika Chaudhuri

Data poisoning attacks -- where an adversary can modify a small fraction of training data, with the goal of forcing the trained classifier to high loss -- are an important threat for machine learning in many applications.

Data Poisoning General Classification

Rearchitecting Classification Frameworks For Increased Robustness

no code implementations26 May 2019 Varun Chandrasekaran, Brian Tang, Nicolas Papernot, Kassem Fawaz, Somesh Jha, Xi Wu

and how to design a classification paradigm that leverages these invariances to improve the robustness accuracy trade-off?

Autonomous Driving Classification +1

Robust Attribution Regularization

1 code implementation NeurIPS 2019 Jiefeng Chen, Xi Wu, Vaibhav Rastogi, YIngyu Liang, Somesh Jha

An emerging problem in trustworthy machine learning is to train models that produce robust interpretations for their predictions.

Privacy-Preserving Collaborative Prediction using Random Forests

no code implementations21 Nov 2018 Irene Giacomelli, Somesh Jha, Ross Kleiman, David Page, Kyonghwan Yoon

We study the problem of privacy-preserving machine learning (PPML) for ensemble methods, focusing our effort on random forests.

Privacy Preserving

Exploring Connections Between Active Learning and Model Extraction

no code implementations5 Nov 2018 Varun Chandrasekaran, Kamalika Chaudhuri, Irene Giacomelli, Somesh Jha, Songbai Yan

This has resulted in the surge of Machine Learning-as-a-Service (MLaaS) - cloud services that provide (a) tools and resources to learn the model, and (b) a user-friendly query interface to access the model.

Active Learning BIG-bench Machine Learning +1

Concise Explanations of Neural Networks using Adversarial Training

1 code implementation ICML 2020 Prasad Chalasani, Jiefeng Chen, Amrita Roy Chowdhury, Somesh Jha, Xi Wu

Our first contribution is a theoretical exploration of how these two properties (when using attributions based on Integrated Gradients, or IG) are related to adversarial training, for a class of 1-layer networks (which includes logistic regression models for binary and multi-class classification); for these networks we show that (a) adversarial training using an $\ell_\infty$-bounded adversary produces models with sparse attribution vectors, and (b) natural model-training while encouraging stable explanations (via an extra term in the loss function), is equivalent to adversarial training.

Multi-class Classification

Explainable Black-Box Attacks Against Model-based Authentication

no code implementations28 Sep 2018 Washington Garcia, Joseph I. Choi, Suman K. Adari, Somesh Jha, Kevin R. B. Butler

Establishing unique identities for both humans and end systems has been an active research problem in the security community, giving rise to innovative machine learning-based authentication techniques.

BIG-bench Machine Learning Explainable Artificial Intelligence (XAI)

Neural-Augmented Static Analysis of Android Communication

no code implementations11 Sep 2018 Jinman Zhao, Aws Albarghouthi, Vaibhav Rastogi, Somesh Jha, Damien Octeau

We address the problem of discovering communication links between applications in the popular Android mobile operating system, an important problem for security and privacy in Android.

Towards Understanding Limitations of Pixel Discretization Against Adversarial Attacks

1 code implementation20 May 2018 Jiefeng Chen, Xi Wu, Vaibhav Rastogi, YIngyu Liang, Somesh Jha

We analyze our results in a theoretical framework and offer strong evidence that pixel discretization is unlikely to work on all but the simplest of the datasets.

Semantic Adversarial Deep Learning

no code implementations19 Apr 2018 Tommaso Dreossi, Somesh Jha, Sanjit A. Seshia

However, existing approaches to generating adversarial examples and devising robust ML algorithms mostly ignore the semantics and context of the overall system containing the ML component.

Malware Detection Self-Driving Cars

The Manifold Assumption and Defenses Against Adversarial Perturbations

no code implementations ICLR 2018 Xi Wu, Uyeong Jang, Lingjiao Chen, Somesh Jha

Interestingly, we find that a recent objective by Madry et al. encourages training a model that satisfies well our formal version of the goodness property, but has a weak control of points that are wrong but with low confidence.

Reinforcing Adversarial Robustness using Model Confidence Induced by Adversarial Training

no code implementations ICML 2018 Xi Wu, Uyeong Jang, Jiefeng Chen, Lingjiao Chen, Somesh Jha

In this paper we study leveraging confidence information induced by adversarial training to reinforce adversarial robustness of a given adversarially trained model.

Adversarial Robustness

Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting

1 code implementation5 Sep 2017 Samuel Yeom, Irene Giacomelli, Matt Fredrikson, Somesh Jha

This paper examines the effect that overfitting and influence have on the ability of an attacker to learn information about the training data from machine learning models, either through training set membership inference or attribute inference attacks.

BIG-bench Machine Learning

Analyzing the Robustness of Nearest Neighbors to Adversarial Examples

1 code implementation ICML 2018 Yizhen Wang, Somesh Jha, Kamalika Chaudhuri

Our analysis shows that its robustness properties depend critically on the value of k - the classifier may be inherently non-robust for small k, but its robustness approaches that of the Bayes Optimal classifier for fast-growing k. We propose a novel modified 1-nearest neighbor classifier, and guarantee its robustness in the large sample limit.

Bolt-on Differential Privacy for Scalable Stochastic Gradient Descent-based Analytics

1 code implementation15 Jun 2016 Xi Wu, Fengan Li, Arun Kumar, Kamalika Chaudhuri, Somesh Jha, Jeffrey F. Naughton

This paper takes a first step to remedy this disconnect and proposes a private SGD algorithm to address \emph{both} issues in an integrated manner.

Practical Black-Box Attacks against Machine Learning

17 code implementations8 Feb 2016 Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, Ananthram Swami

Our attack strategy consists in training a local model to substitute for the target DNN, using inputs synthetically generated by an adversary and labeled by the target DNN.

BIG-bench Machine Learning

Revisiting Differentially Private Regression: Lessons From Learning Theory and their Consequences

no code implementations20 Dec 2015 Xi Wu, Matthew Fredrikson, Wentao Wu, Somesh Jha, Jeffrey F. Naughton

Perhaps more importantly, our theory reveals that the most basic mechanism in differential privacy, output perturbation, can be used to obtain a better tradeoff for all convex-Lipschitz-bounded learning tasks.

Learning Theory regression

The Limitations of Deep Learning in Adversarial Settings

11 code implementations24 Nov 2015 Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, Ananthram Swami

In this work, we formalize the space of adversaries against deep neural networks (DNNs) and introduce a novel class of algorithms to craft adversarial samples based on a precise understanding of the mapping between inputs and outputs of DNNs.

Adversarial Attack Adversarial Defense

Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks

2 code implementations14 Nov 2015 Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, Ananthram Swami

In this work, we introduce a defensive mechanism called defensive distillation to reduce the effectiveness of adversarial samples on DNNs.

Autonomous Vehicles BIG-bench Machine Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.