Search Results for author:
Found 32 papers, 17 papers with code
Transfer Attacks and Defenses for Large Language Models on Coding Tasks
They improve upon previous neural network models of code, such as code2seq or seq2seq, that already demonstrated competitive results when performing tasks such as code summarization and identifying code vulnerabilities.
Is Certifying $\ell_p$ Robustness Still Worthwhile?
There are three layers to this inquiry, which we address in this paper: (1) why do we care about robustness research?
A Recipe for Improved Certifiable Robustness: Capacity and Data
A key challenge, supported both theoretically and empirically, is that robustness demands greater network capacity and more data than standard training.
Representation Engineering: A Top-Down Approach to AI Transparency
In this paper, we identify and characterize the emerging area of representation engineering (RepE), an approach to enhancing the transparency of AI systems that draws on insights from cognitive neuroscience.
Ranked #3 on
Question Answering
on TruthfulQA
Universal and Transferable Adversarial Attacks on Aligned Language Models
Specifically, our approach finds a suffix that, when attached to a wide range of queries for an LLM to produce objectionable content, aims to maximize the probability that the model produces an affirmative response (rather than refusing to answer).
Unlocking Deterministic Robustness Certification on ImageNet
We show that fast ways of bounding the Lipschitz constant for conventional ResNets are loose, and show how to address this by designing a new residual block, leading to the \emph{Linear ResNet} (LiResNet) architecture.
Learning Modulo Theories
Recent techniques that integrate \emph{solver layers} into Deep Neural Networks (DNNs) have shown promise in bridging a long-standing gap between inductive learning and symbolic reasoning techniques.
Black-Box Audits for Group Distribution Shifts
Our experimental results on real-world datasets show that this approach is effective, achieving 80--100% AUC-ROC in detecting shifts involving the underrepresentation of a demographic group in the training set.
On the Perils of Cascading Robust Classifiers
We present \emph{cascade attack} (CasA), an adversarial attack against cascading ensembles, and show that: (1) there exists an adversarial input for up to 88\% of the samples where the ensemble claims to be certifiably robust and accurate; and (2) the accuracy of a cascading ensemble under our attack is as low as 11\% when it claims to be certifiably robust and accurate on 97\% of the test set.
Faithful Explanations for Deep Graph Models
Second, our analytical and empirical results demonstrate that feature attribution methods cannot capture the nonlinear effect of edge features, while existing subgraph explanation methods are not faithful.
Selective Ensembles for Consistent Predictions
Recent work has shown that models trained to the same objective, and which achieve similar measures of accuracy on consistent test data, may nonetheless behave very differently on individual predictions.
Consistent Counterfactuals for Deep Models
Counterfactual examples are one of the most commonly-cited methods for explaining the predictions of machine learning models in key areas such as finance and medical diagnosis.
Degradation Attacks on Certifiably Robust Neural Networks
Certifiably robust neural networks employ provable run-time defenses against adversarial examples by checking if the model is locally robust at the input under evaluation.
Self-Correcting Neural Networks For Safe Classification
These constraints relate requirements on the order of the classes output by a classifier to conditions on its input, and are expressive enough to encode various interesting examples of classifier safety specifications from the literature.
Leave-one-out Unfairness
We introduce leave-one-out unfairness, which characterizes how likely a model's prediction for an individual will change due to the inclusion or removal of a single other person in the model's training data.
Relaxing Local Robustness
Certifiable local robustness, which rigorously precludes small-norm adversarial examples, has received significant attention as a means of addressing security concerns in deep learning.
Robust Models Are More Interpretable Because Attributions Look Normal
Recent work has found that adversarially-robust deep networks used for image classification are more interpretable: their feature attributions tend to be sharper, and are more concentrated on the objects associated with the image's ground-truth class.
Globally-Robust Neural Networks
We show that widely-used architectures can be easily adapted to this objective by incorporating efficient global Lipschitz bounds into the network, yielding certifiably-robust models by construction that achieve state-of-the-art verifiable accuracy.
Smoothed Geometry for Robust Attribution
Feature attributions are a popular tool for explaining the behavior of Deep Neural Networks (DNNs), but have recently been shown to be vulnerable to attacks that produce divergent explanations for nearby inputs.
Interpreting Interpretations: Organizing Attribution Methods by Criteria
In this work we expand the foundationsof human-understandable concepts with which attributionscan be interpreted beyond "importance" and its visualization; we incorporate the logical concepts of necessity andsufficiency, and the concept of proportionality.
Individual Fairness Revisited: Transferring Techniques from Adversarial Robustness
We turn the definition of individual fairness on its head---rather than ascertaining the fairness of a model given a predetermined metric, we find a metric for a given model that satisfies individual fairness.
Fast Geometric Projections for Local Robustness Certification
Local robustness ensures that a model classifies all inputs within an $\ell_2$-ball consistently, which precludes various forms of adversarial inputs.
Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inference
Membership inference (MI) attacks exploit the fact that machine learning algorithms sometimes leak information about their training data through the learned model.
Learning Fair Representations for Kernel Models
In contrast, we demonstrate the promise of learning a model-aware fair representation, focusing on kernel-based models.
FlipTest: Fairness Testing via Optimal Transport
We present FlipTest, a black-box technique for uncovering discrimination in classifiers.
Feature-Wise Bias Amplification
This overestimation gives rise to feature-wise bias amplification -- a previously unreported form of bias that can be traced back to the features of a trained model.
Hunting for Discriminatory Proxies in Linear Regression Models
In this paper we formulate a definition of proxy use for the setting of linear regression and present algorithms for detecting proxies.
Influence-Directed Explanations for Deep Convolutional Networks
We study the problem of explaining a rich class of behavioral properties of deep neural networks.
Case Study: Explaining Diabetic Retinopathy Detection Deep CNNs via Integrated Gradients
In this report, we applied integrated gradients to explaining a neural network for diabetic retinopathy detection.
Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting
This paper examines the effect that overfitting and influence have on the ability of an attacker to learn information about the training data from machine learning models, either through training set membership inference or attribute inference attacks.
Proxy Non-Discrimination in Data-Driven Systems
Machine learnt systems inherit biases against protected classes, historically disparaged groups, from training data.
The Limitations of Deep Learning in Adversarial Settings
In this work, we formalize the space of adversaries against deep neural networks (DNNs) and introduce a novel class of algorithms to craft adversarial samples based on a precise understanding of the mapping between inputs and outputs of DNNs.
