Search Results for author: Somesh Jha

Found 67 papers, 23 papers with code

Constraining the Attack Space of Machine Learning Models with Distribution Clamping Preprocessing

no code implementations18 May 2022 Ryan Feng, Somesh Jha, Atul Prakash

Preprocessing and outlier detection techniques have both been applied to neural networks to increase robustness with varying degrees of success.

Object Detection Outlier Detection

Optimal Membership Inference Bounds for Adaptive Composition of Sampled Gaussian Mechanisms

no code implementations12 Apr 2022 Saeed Mahloujifar, Alexandre Sablayrolles, Graham Cormode, Somesh Jha

A common countermeasure against MI attacks is to utilize differential privacy (DP) during model training to mask the presence of individual examples.

Concept-based Explanations for Out-Of-Distribution Detectors

no code implementations4 Mar 2022 Jihye Choi, Jayaram Raghuram, Ryan Feng, Jiefeng Chen, Somesh Jha, Atul Prakash

Based on these metrics, we propose a framework for learning a set of concepts that satisfy the desired properties of detection completeness and concept separability and demonstrate the framework's effectiveness in providing concept-based explanations for diverse OOD techniques.

OOD Detection

A Quantitative Geometric Approach to Neural Network Smoothness

no code implementations2 Mar 2022 Zi Wang, Gautam Prakriya, Somesh Jha

In this work, we provide a unified theoretical framework, a quantitative geometric approach, to address the Lipschitz constant estimation.

Towards Adversarially Robust Deepfake Detection: An Ensemble Approach

no code implementations11 Feb 2022 Ashish Hooda, Neal Mangaokar, Ryan Feng, Kassem Fawaz, Somesh Jha, Atul Prakash

Detecting deepfakes is an important problem, but recent work has shown that DNN-based deepfake detectors are brittle against adversarial deepfakes, in which an adversary adds imperceptible perturbations to a deepfake to evade detection.

DeepFake Detection Face Swapping

An Exploration of Multicalibration Uniform Convergence Bounds

no code implementations9 Feb 2022 Harrison Rosenberg, Robi Bhattacharjee, Kassem Fawaz, Somesh Jha

Given the prevalence of ERM sample complexity bounds, our proposed framework enables machine learning practitioners to easily understand the convergence behavior of multicalibration error for a myriad of classifier architectures.

Fairness

Revisiting Adversarial Robustness of Classifiers With a Reject Option

no code implementations AAAI Workshop AdvML 2022 Jiefeng Chen, Jayaram Raghuram, Jihye Choi, Xi Wu, YIngyu Liang, Somesh Jha

Motivated by this metric, we propose novel loss functions and a robust training method -- \textit{stratified adversarial training with rejection} (SATR) -- for a classifier with reject option, where the goal is to accept and correctly-classify small input perturbations, while allowing the rejection of larger input perturbations that cannot be correctly classified.

Adversarial Robustness Image Classification

Towards Evaluating the Robustness of Neural Networks Learned by Transduction

1 code implementation ICLR 2022 Jiefeng Chen, Xi Wu, Yang Guo, YIngyu Liang, Somesh Jha

There has been emerging interest in using transductive learning for adversarial robustness (Goldwasser et al., NeurIPS 2020; Wu et al., ICML 2020; Wang et al., ArXiv 2021).

Adversarial Robustness Bilevel Optimization

Fast and Sample-Efficient Domain Adaptation for Autoencoder-Based End-to-End Communication

no code implementations29 Sep 2021 Jayaram Raghuram, Yijing Zeng, Dolores Garcia, Somesh Jha, Suman Banerjee, Joerg Widmer, Rafael Ruiz

In this paper, we address the setting where the target domain has only limited labeled data from a distribution that is expected to change frequently.

Domain Adaptation

Privacy Implications of Shuffling

no code implementations ICLR 2022 Casey Meehan, Amrita Roy Chowdhury, Kamalika Chaudhuri, Somesh Jha

\ldp deployments are vulnerable to inference attacks as an adversary can link the noisy responses to their identity and subsequently, auxiliary information using the \textit{order} of the data.

Less is More: Dimension Reduction Finds On-Manifold Adversarial Examples in Hard-Label Attacks

no code implementations29 Sep 2021 Washington Garcia, Pin-Yu Chen, Somesh Jha, Hamilton Scott Clouse, Kevin R. B. Butler

It was recently shown in the gradient-level setting that regular adversarial examples leave the data manifold, while their on-manifold counterparts are in fact generalization errors.

Dimensionality Reduction Image Classification

Fairness Properties of Face Recognition and Obfuscation Systems

no code implementations5 Aug 2021 Harrison Rosenberg, Brian Tang, Kassem Fawaz, Somesh Jha

A recent, popular approach to address these privacy concerns is to employ evasion attacks against the metric embedding networks powering face recognition systems.

Face Recognition Fairness

Domain Adaptation for Autoencoder-Based End-to-End Communication Over Wireless Channels

no code implementations2 Aug 2021 Jayaram Raghuram, Yijing Zeng, Dolores García Martí, Somesh Jha, Suman Banerjee, Joerg Widmer, Rafael Ruiz Ortiz

We propose a method for adapting the autoencoder without modifying the encoder and decoder neural networks, and adapting only the MDN model of the channel.

Domain Adaptation

Detecting Errors and Estimating Accuracy on Unlabeled Data with Self-training Ensembles

1 code implementation NeurIPS 2021 Jiefeng Chen, Frederick Liu, Besim Avci, Xi Wu, YIngyu Liang, Somesh Jha

This observation leads to two challenging tasks: (1) unsupervised accuracy estimation, which aims to estimate the accuracy of a pre-trained classifier on a set of unlabeled test inputs; (2) error detection, which aims to identify mis-classified test inputs.

Towards Adversarial Robustness via Transductive Learning

no code implementations15 Jun 2021 Jiefeng Chen, Yang Guo, Xi Wu, Tianqi Li, Qicheng Lao, YIngyu Liang, Somesh Jha

Compared to traditional "test-time" defenses, these defense mechanisms "dynamically retrain" the model based on test time input via transductive learning; and theoretically, attacking these defenses boils down to bilevel optimization, which seems to raise the difficulty for adaptive attacks.

Adversarial Robustness Bilevel Optimization

A Shuffling Framework for Local Differential Privacy

no code implementations11 Jun 2021 Casey Meehan, Amrita Roy Chowdhury, Kamalika Chaudhuri, Somesh Jha

ldp deployments are vulnerable to inference attacks as an adversary can link the noisy responses to their identity and subsequently, auxiliary information using the order of the data.

Hard-label Manifolds: Unexpected Advantages of Query Efficiency for Finding On-manifold Adversarial Examples

no code implementations4 Mar 2021 Washington Garcia, Pin-Yu Chen, Somesh Jha, Scott Clouse, Kevin R. B. Butler

It was recently shown in the gradient-level setting that regular adversarial examples leave the data manifold, while their on-manifold counterparts are in fact generalization errors.

Dimensionality Reduction Image Classification

Exploring Adversarial Robustness of Deep Metric Learning

1 code implementation14 Feb 2021 Thomas Kobber Panum, Zi Wang, Pengyu Kan, Earlence Fernandes, Somesh Jha

Deep Metric Learning (DML), a widely-used technique, involves learning a distance metric between pairs of samples.

Adversarial Robustness Metric Learning

CaPC Learning: Confidential and Private Collaborative Learning

1 code implementation ICLR 2021 Christopher A. Choquette-Choo, Natalie Dullerud, Adam Dziedzic, Yunxiang Zhang, Somesh Jha, Nicolas Papernot, Xiao Wang

There is currently no method that enables machine learning in such a setting, where both confidentiality and privacy need to be preserved, to prevent both explicit and implicit sharing of data.

Fairness Federated Learning

Generalized Universal Approximation for Certified Networks

no code implementations1 Jan 2021 Zi Wang, Aws Albarghouthi, Somesh Jha

To certify safety and robustness of neural networks, researchers have successfully applied abstract interpretation, primarily using interval bound propagation.

Test-Time Adaptation and Adversarial Robustness

no code implementations1 Jan 2021 Xi Wu, Yang Guo, Tianqi Li, Jiefeng Chen, Qicheng Lao, YIngyu Liang, Somesh Jha

On the positive side, we show that, if one is allowed to access the training data, then Domain Adversarial Neural Networks (${\sf DANN}$), an algorithm designed for unsupervised domain adaptation, can provide nontrivial robustness in the test-time maximin threat model against strong transfer attacks and adaptive fixed point attacks.

Adversarial Robustness Unsupervised Domain Adaptation

Adversarial Deep Metric Learning

no code implementations1 Jan 2021 Thomas Kobber Panum, Zi Wang, Pengyu Kan, Earlence Fernandes, Somesh Jha

To the best of our knowledge, we are the first to systematically analyze this dependence effect and propose a principled approach for robust training of deep metric learning networks that accounts for the nuances of metric losses.

Metric Learning

Sample Complexity of Adversarially Robust Linear Classification on Separated Data

no code implementations19 Dec 2020 Robi Bhattacharjee, Somesh Jha, Kamalika Chaudhuri

This shows that for very well-separated data, convergence rates of $O(\frac{1}{n})$ are achievable, which is not the case otherwise.

Adversarial Robustness Classification +1

ShadowNet: A Secure and Efficient System for On-device Model Inference

no code implementations11 Nov 2020 Zhichuang Sun, Ruimin Sun, Changming Liu, Amrita Roy Chowdhury, Somesh Jha, Long Lu

ShadowNet protects the model privacy with Trusted Execution Environment (TEE) while securely outsourcing the heavy linear layers of the model to the untrusted hardware accelerators.

Is Private Learning Possible with Instance Encoding?

1 code implementation10 Nov 2020 Nicholas Carlini, Samuel Deng, Sanjam Garg, Somesh Jha, Saeed Mahloujifar, Mohammad Mahmoody, Shuang Song, Abhradeep Thakurta, Florian Tramer

A private machine learning algorithm hides as much as possible about its training data while still preserving accuracy.

Informative Outlier Matters: Robustifying Out-of-distribution Detection Using Outlier Mining

no code implementations28 Sep 2020 Jiefeng Chen, Yixuan Li, Xi Wu, YIngyu Liang, Somesh Jha

We show that, by mining informative auxiliary OOD data, one can significantly improve OOD detection performance, and somewhat surprisingly, generalize to unseen adversarial attacks.

OOD Detection Out-of-Distribution Detection

A General Framework For Detecting Anomalous Inputs to DNN Classifiers

1 code implementation29 Jul 2020 Jayaram Raghuram, Varun Chandrasekaran, Somesh Jha, Suman Banerjee

We propose an unsupervised anomaly detection framework based on the internal DNN layer representations in the form of a meta-algorithm with configurable components.

Image Classification Unsupervised Anomaly Detection

Interval Universal Approximation for Neural Networks

no code implementations12 Jul 2020 Zi Wang, Aws Albarghouthi, Gautam Prakriya, Somesh Jha

This is a crucial question, as our constructive proof of IUA is exponential in the size of the approximation domain.

Robust and Accurate Authorship Attribution via Program Normalization

no code implementations1 Jul 2020 Yizhen Wang, Mohannad Alhanahnah, Ke Wang, Mihai Christodorescu, Somesh Jha

To address these emerging issues, we formulate this security challenge into a general threat model, the $\textit{relational adversary}$, that allows an arbitrary number of the semantics-preserving transformations to be applied to an input in any problem space.

Image Classification Malware Detection

ATOM: Robustifying Out-of-distribution Detection Using Outlier Mining

1 code implementation26 Jun 2020 Jiefeng Chen, Yixuan Li, Xi Wu, YIngyu Liang, Somesh Jha

We show that, by mining informative auxiliary OOD data, one can significantly improve OOD detection performance, and somewhat surprisingly, generalize to unseen adversarial attacks.

OOD Detection Out-of-Distribution Detection

Representation Bayesian Risk Decompositions and Multi-Source Domain Adaptation

no code implementations22 Apr 2020 Xi Wu, Yang Guo, Jiefeng Chen, YIngyu Liang, Somesh Jha, Prasad Chalasani

Recent studies provide hints and failure examples for domain invariant representation learning, a common approach for this problem, but the explanations provided are somewhat different and do not provide a unified picture.

Domain Adaptation Representation Learning

Robust Out-of-distribution Detection for Neural Networks

1 code implementation AAAI Workshop AdvML 2022 Jiefeng Chen, Yixuan Li, Xi Wu, YIngyu Liang, Somesh Jha

Formally, we extensively study the problem of Robust Out-of-Distribution Detection on common OOD detection approaches, and show that state-of-the-art OOD detectors can be easily fooled by adding small perturbations to the in-distribution and OOD inputs.

OOD Detection Out-of-Distribution Detection

Face-Off: Adversarial Face Obfuscation

1 code implementation19 Mar 2020 Chuhan Gao, Varun Chandrasekaran, Kassem Fawaz, Somesh Jha

We implement and evaluate Face-Off to find that it deceives three commercial face recognition services from Microsoft, Amazon, and Face++.

Cryptography and Security

Analyzing Accuracy Loss in Randomized Smoothing Defenses

no code implementations3 Mar 2020 Yue Gao, Harrison Rosenberg, Kassem Fawaz, Somesh Jha, Justin Hsu

In test-time attacks an adversary crafts adversarial examples, which are specially crafted perturbations imperceptible to humans which, when added to an input example, force a machine learning model to misclassify the given input example.

Autonomous Driving Speech Recognition

CAUSE: Learning Granger Causality from Event Sequences using Attribution Methods

1 code implementation ICML 2020 Wei Zhang, Thomas Kobber Panum, Somesh Jha, Prasad Chalasani, David Page

We study the problem of learning Granger causality between event types from asynchronous, interdependent, multi-type event sequences.

GRAPHITE: Generating Automatic Physical Examples for Machine-Learning Attacks on Computer Vision Systems

1 code implementation17 Feb 2020 Ryan Feng, Neal Mangaokar, Jiefeng Chen, Earlence Fernandes, Somesh Jha, Atul Prakash

We address three key requirements for practical attacks for the real-world: 1) automatically constraining the size and shape of the attack so it can be applied with stickers, 2) transform-robustness, i. e., robustness of a attack to environmental physical variations such as viewpoint and lighting changes, and 3) supporting attacks in not only white-box, but also black-box hard-label scenarios, so that the adversary can attack proprietary models.

General Classification License Plate Recognition

Semantic Robustness of Models of Source Code

1 code implementation7 Feb 2020 Goutham Ramakrishnan, Jordan Henkel, Zi Wang, Aws Albarghouthi, Somesh Jha, Thomas Reps

Deep neural networks are vulnerable to adversarial examples - small input perturbations that result in incorrect predictions.

Generating Semantic Adversarial Examples with Differentiable Rendering

no code implementations2 Oct 2019 Lakshya Jain, Wilson Wu, Steven Chen, Uyeong Jang, Varun Chandrasekaran, Sanjit Seshia, Somesh Jha

In this paper we explore semantic adversarial examples (SAEs) where an attacker creates perturbations in the semantic space representing the environment that produces input for the ML model.

Autonomous Driving

On the Need for Topology-Aware Generative Models for Manifold-Based Defenses

no code implementations ICLR 2020 Uyeong Jang, Susmit Jha, Somesh Jha

These defenses rely on the assumption that data lie in a manifold of a lower dimension than the input space.

Data Augmentation

Improving Utility and Security of the Shuffler-based Differential Privacy

1 code implementation30 Aug 2019 Tianhao Wang, Bolin Ding, Min Xu, Zhicong Huang, Cheng Hong, Jingren Zhou, Ninghui Li, Somesh Jha

When collecting information, local differential privacy (LDP) alleviates privacy concerns of users because their private information is randomized before being sent it to the central aggregator.

Data-Dependent Differentially Private Parameter Learning for Directed Graphical Models

no code implementations ICML 2020 Amrita Roy Chowdhury, Theodoros Rekatsinas, Somesh Jha

Our solution optimizes for the utility of inference queries over the DGM and \textit{adds noise that is customized to the properties of the private input dataset and the graph structure of the DGM}.

Adversarially Robust Learning Could Leverage Computational Hardness

no code implementations28 May 2019 Sanjam Garg, Somesh Jha, Saeed Mahloujifar, Mohammad Mahmoody

On the reverse directions, we also show that the existence of such learning task in which computational robustness beats information theoretic robustness requires computational hardness by implying (average-case) hardness of NP.

An Investigation of Data Poisoning Defenses for Online Learning

no code implementations28 May 2019 Yizhen Wang, Somesh Jha, Kamalika Chaudhuri

Data poisoning attacks -- where an adversary can modify a small fraction of training data, with the goal of forcing the trained classifier to high loss -- are an important threat for machine learning in many applications.

Data Poisoning General Classification +1

Rearchitecting Classification Frameworks For Increased Robustness

no code implementations26 May 2019 Varun Chandrasekaran, Brian Tang, Nicolas Papernot, Kassem Fawaz, Somesh Jha, Xi Wu

and how to design a classification paradigm that leverages these invariances to improve the robustness accuracy trade-off?

Autonomous Driving Classification +1

Robust Attribution Regularization

1 code implementation NeurIPS 2019 Jiefeng Chen, Xi Wu, Vaibhav Rastogi, YIngyu Liang, Somesh Jha

An emerging problem in trustworthy machine learning is to train models that produce robust interpretations for their predictions.

Privacy-Preserving Collaborative Prediction using Random Forests

no code implementations21 Nov 2018 Irene Giacomelli, Somesh Jha, Ross Kleiman, David Page, Kyonghwan Yoon

We study the problem of privacy-preserving machine learning (PPML) for ensemble methods, focusing our effort on random forests.

Exploring Connections Between Active Learning and Model Extraction

no code implementations5 Nov 2018 Varun Chandrasekaran, Kamalika Chaudhuri, Irene Giacomelli, Somesh Jha, Songbai Yan

This has resulted in the surge of Machine Learning-as-a-Service (MLaaS) - cloud services that provide (a) tools and resources to learn the model, and (b) a user-friendly query interface to access the model.

Active Learning Model extraction

Concise Explanations of Neural Networks using Adversarial Training

1 code implementation ICML 2020 Prasad Chalasani, Jiefeng Chen, Amrita Roy Chowdhury, Somesh Jha, Xi Wu

Our first contribution is a theoretical exploration of how these two properties (when using attributions based on Integrated Gradients, or IG) are related to adversarial training, for a class of 1-layer networks (which includes logistic regression models for binary and multi-class classification); for these networks we show that (a) adversarial training using an $\ell_\infty$-bounded adversary produces models with sparse attribution vectors, and (b) natural model-training while encouraging stable explanations (via an extra term in the loss function), is equivalent to adversarial training.

Multi-class Classification

Explainable Black-Box Attacks Against Model-based Authentication

no code implementations28 Sep 2018 Washington Garcia, Joseph I. Choi, Suman K. Adari, Somesh Jha, Kevin R. B. Butler

Establishing unique identities for both humans and end systems has been an active research problem in the security community, giving rise to innovative machine learning-based authentication techniques.

Neural-Augmented Static Analysis of Android Communication

no code implementations11 Sep 2018 Jinman Zhao, Aws Albarghouthi, Vaibhav Rastogi, Somesh Jha, Damien Octeau

We address the problem of discovering communication links between applications in the popular Android mobile operating system, an important problem for security and privacy in Android.

Towards Understanding Limitations of Pixel Discretization Against Adversarial Attacks

1 code implementation20 May 2018 Jiefeng Chen, Xi Wu, Vaibhav Rastogi, YIngyu Liang, Somesh Jha

We analyze our results in a theoretical framework and offer strong evidence that pixel discretization is unlikely to work on all but the simplest of the datasets.

Semantic Adversarial Deep Learning

no code implementations19 Apr 2018 Tommaso Dreossi, Somesh Jha, Sanjit A. Seshia

However, existing approaches to generating adversarial examples and devising robust ML algorithms mostly ignore the semantics and context of the overall system containing the ML component.

Malware Detection Self-Driving Cars

The Manifold Assumption and Defenses Against Adversarial Perturbations

no code implementations ICLR 2018 Xi Wu, Uyeong Jang, Lingjiao Chen, Somesh Jha

Interestingly, we find that a recent objective by Madry et al. encourages training a model that satisfies well our formal version of the goodness property, but has a weak control of points that are wrong but with low confidence.

Reinforcing Adversarial Robustness using Model Confidence Induced by Adversarial Training

no code implementations ICML 2018 Xi Wu, Uyeong Jang, Jiefeng Chen, Lingjiao Chen, Somesh Jha

In this paper we study leveraging confidence information induced by adversarial training to reinforce adversarial robustness of a given adversarially trained model.

Adversarial Robustness

Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting

1 code implementation5 Sep 2017 Samuel Yeom, Irene Giacomelli, Matt Fredrikson, Somesh Jha

This paper examines the effect that overfitting and influence have on the ability of an attacker to learn information about the training data from machine learning models, either through training set membership inference or attribute inference attacks.

Analyzing the Robustness of Nearest Neighbors to Adversarial Examples

1 code implementation ICML 2018 Yizhen Wang, Somesh Jha, Kamalika Chaudhuri

Our analysis shows that its robustness properties depend critically on the value of k - the classifier may be inherently non-robust for small k, but its robustness approaches that of the Bayes Optimal classifier for fast-growing k. We propose a novel modified 1-nearest neighbor classifier, and guarantee its robustness in the large sample limit.

Bolt-on Differential Privacy for Scalable Stochastic Gradient Descent-based Analytics

1 code implementation15 Jun 2016 Xi Wu, Fengan Li, Arun Kumar, Kamalika Chaudhuri, Somesh Jha, Jeffrey F. Naughton

This paper takes a first step to remedy this disconnect and proposes a private SGD algorithm to address \emph{both} issues in an integrated manner.

Practical Black-Box Attacks against Machine Learning

17 code implementations8 Feb 2016 Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, Ananthram Swami

Our attack strategy consists in training a local model to substitute for the target DNN, using inputs synthetically generated by an adversary and labeled by the target DNN.

Revisiting Differentially Private Regression: Lessons From Learning Theory and their Consequences

no code implementations20 Dec 2015 Xi Wu, Matthew Fredrikson, Wentao Wu, Somesh Jha, Jeffrey F. Naughton

Perhaps more importantly, our theory reveals that the most basic mechanism in differential privacy, output perturbation, can be used to obtain a better tradeoff for all convex-Lipschitz-bounded learning tasks.

Learning Theory

The Limitations of Deep Learning in Adversarial Settings

12 code implementations24 Nov 2015 Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, Ananthram Swami

In this work, we formalize the space of adversaries against deep neural networks (DNNs) and introduce a novel class of algorithms to craft adversarial samples based on a precise understanding of the mapping between inputs and outputs of DNNs.

Adversarial Attack Adversarial Defense

Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks

1 code implementation14 Nov 2015 Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, Ananthram Swami

In this work, we introduce a defensive mechanism called defensive distillation to reduce the effectiveness of adversarial samples on DNNs.

Autonomous Vehicles

Cannot find the paper you are looking for? You can Submit a new open access paper.