Search Results for author:
Found 114 papers, 53 papers with code
RESTOR: Knowledge Recovery through Machine Unlearning
Large language models trained on web-scale corpora can memorize undesirable datapoints such as incorrect facts, copyrighted content or sensitive data.
Unearthing Skill-Level Insights for Understanding Trade-Offs of Foundation Models
With models getting stronger, evaluations have grown more complex, testing multiple skills in one benchmark and even in the same instance at once.
IntCoOp: Interpretability-Aware Vision-Language Prompt Tuning
Notably, in a 16-shot setup, IntCoOp improves CoOp by 7. 35% in average performance across 10 diverse datasets.
Endor: Hardware-Friendly Sparse Format for Offloaded LLM Inference
In our case study of offloaded inference, we found that due to the low bandwidth between storage devices and GPU, the latency of transferring large model weights from its offloaded location to GPU memory becomes the critical bottleneck with actual compute taking nearly 0% of runtime.
Understanding and Mitigating Compositional Issues in Text-to-Image Generative Models
We also show that re-weighting the erroneous attention contributions in CLIP can also lead to improved compositional performances, however these improvements are often less significant than those achieved by solely learning a linear projection head, highlighting erroneous attentions to be only a minor error source.
Understanding Information Storage and Transfer in Multi-modal Large Language Models
Understanding the mechanisms of information storage and transfer in Transformer-based models is important for driving model understanding progress.
DREW : Towards Robust Data Provenance by Leveraging Error-Controlled Watermarking
Identifying the origin of data is crucial for data provenance, with applications including data ownership protection, media forensics, and detecting AI-generated content.
Loki: Low-rank Keys for Efficient Sparse Attention
Inference on large language models (LLMs) can be expensive in terms of the compute and memory costs involved, especially when long sequence lengths are used.
Decomposing and Interpreting Image Representations via Text in ViTs Beyond CLIP
To this end, we introduce a general framework which can identify the roles of various components in ViTs beyond CLIP.
Understanding the Effect of using Semantically Meaningful Tokens for Visual Representation Learning
Vision transformers have established a precedent of patchifying images into uniformly-sized chunks before processing.
Securing the Future of GenAI: Policy and Technology
The interplay between legislation and technology is a very vast topic, and we don't claim that this paper is a comprehensive treatment on this topic.
On Mechanistic Knowledge Localization in Text-to-Image Generative Models
To address this issue, we introduce the concept of Mechanistic Localization in text-to-image models, where knowledge about various visual attributes (e. g., "style", "objects", "facts") can be mechanistically localized to a small fraction of layers in the UNet, thus facilitating efficient model editing.
Rethinking Artistic Copyright Infringements in the Era of Text-to-Image Generative Models
Recent text-to-image generative models such as Stable Diffusion are extremely adept at mimicking and generating copyrighted content, raising concerns amongst artists that their unique styles may be improperly copied.
What do we learn from inverting CLIP models?
We employ an inversion-based approach to examine CLIP models.
Fast Adversarial Attacks on Language Models In One GPU Minute
Through human evaluations, we find that our untargeted attack causes Vicuna-7B-v1. 5 to produce ~15% more incorrect outputs when compared to LM outputs in the absence of our attack.
Identifying and Mitigating Model Failures through Few-shot CLIP-aided Diffusion Generation
One common reason for these failures is the occurrence of objects in backgrounds that are rarely seen during training.
Instruct2Attack: Language-Guided Semantic Adversarial Attacks
We propose Instruct2Attack (I2A), a language-guided semantic attack that generates semantically meaningful perturbations according to free-form language instructions.
Online Advertisements with LLMs: Opportunities and Challenges
This paper explores the potential for leveraging Large Language Models (LLM) in the realm of online advertising systems.
Exploring Geometry of Blind Spots in Vision Models
We further observe that the source image is linearly connected by a high-confidence path to these inputs, uncovering a star-like structure for level sets of deep networks.
Localizing and Editing Knowledge in Text-to-Image Generative Models
We adapt Causal Mediation Analysis for text-to-image models and trace knowledge about distinct visual attributes to various (causal) components in the (i) UNet and (ii) text-encoder of the diffusion model.
EditVal: Benchmarking Diffusion Based Text-Guided Image Editing Methods
From both the human study and automated evaluation, we find that: (i) Instruct-Pix2Pix, Null-Text and SINE are the top-performing methods averaged across different edit types, however {\it only} Instruct-Pix2Pix and Null-Text are able to preserve original image properties; (ii) Most of the editing methods fail at edits involving spatial operations (e. g., changing the position of an object).
Robustness of AI-Image Detectors: Fundamental Limits and Practical Attacks
Moreover, we show that watermarking methods are vulnerable to spoofing attacks where the attacker aims to have real images identified as watermarked ones, damaging the reputation of the developers.
PRIME: Prioritizing Interpretability in Failure Mode Extraction
To improve on these shortcomings, we propose a novel approach that prioritizes interpretability in this problem: we start by obtaining human-understandable concepts (tags) of images in the dataset and then analyze the model's behavior based on the presence or absence of combinations of these tags.
Adapting Self-Supervised Representations to Multi-Domain Setups
Current state-of-the-art self-supervised approaches, are effective when trained on individual domains but show limited generalization on unseen domains.
Certifying LLM Safety against Adversarial Prompting
We defend against three attack modes: i) adversarial suffix, where an adversarial sequence is appended at the end of a harmful prompt; ii) adversarial insertion, where the adversarial sequence is inserted anywhere in the middle of the prompt; and iii) adversarial infusion, where adversarial tokens are inserted at arbitrary positions in the prompt, not necessarily as a contiguous block.
Identifying and Mitigating the Security Risks of Generative AI
However, GenAI can be used just as well by attackers to generate new attacks and increase the velocity and efficacy of existing attacks.
Identifying Interpretable Subspaces in Image Representations
Although many existing approaches interpret features independently, we observe in state-of-the-art self-supervised and supervised models, that less than 20% of the representation space can be explained by individual features.
Distilling Knowledge from Text-to-Image Generative Models Improves Visio-Linguistic Reasoning in CLIP
This work underscores the potential of well-designed distillation objectives from generative models to enhance contrastive image-text models with improved visio-linguistic reasoning capabilities.
On Practical Aspects of Aggregation Defenses against Data Poisoning Attacks
The increasing access to data poses both opportunities and risks in deep learning, as one can manipulate the behaviors of deep learning models with malicious training samples.
Text-To-Concept (and Back) via Cross-Model Alignment
We observe that the mapping between an image's representation in one model to its representation in another can be learned surprisingly well with just a linear layer, even across diverse models.
Strong Baselines for Parameter Efficient Few-Shot Fine-tuning
Through our controlled empirical study, we have two main findings: (i) Fine-tuning just the LayerNorm parameters (which we call LN-Tune) during few-shot adaptation is an extremely strong baseline across ViTs pre-trained with both self-supervised and supervised objectives, (ii) For self-supervised ViTs, we find that simply learning a set of scaling parameters for each attention matrix (which we call AttnScale) along with a domain-residual adapter (DRA) module leads to state-of-the-art performance (while being $\sim\!$ 9$\times$ more parameter-efficient) on MD.
Few-Shot Image Classification
parameter-efficient fine-tuning
Provable Robustness for Streaming Models with a Sliding Window
Robustness certificates based on the assumption of independent input samples are not directly applicable in such scenarios.
DRSM: De-Randomized Smoothing on Malware Classifier Providing Certified Robustness
After showing how DRSM is theoretically robust against attacks with contiguous adversarial bytes, we verify its performance and certified robustness experimentally, where we observe only marginal accuracy drops as the cost of robustness.
Can AI-Generated Text be Reliably Detected?
In particular, we develop a recursive paraphrasing attack to apply on AI text, which can break a whole range of detectors, including the ones using the watermarking schemes as well as neural network-based detectors, zero-shot classifiers, and retrieval-based detectors.
CUDA: Convolution-based Unlearnable Datasets
Here, ERM on the clean training data achieves a clean test accuracy of 80. 66$\%$.
Temporal Robustness against Data Poisoning
Data poisoning considers cases when an adversary manipulates the behavior of machine learning algorithms through malicious training data.
Run-Off Election: Improved Provable Defense against Data Poisoning Attacks
Based on this approach, we propose DPA+ROE and FA+ROE defense methods based on Deep Partition Aggregation (DPA) and Finite Aggregation (FA) approaches from prior work.
Towards Improved Input Masking for Convolutional Neural Networks
In this work, we propose a new masking method for CNNs we call layer masking in which the missingness bias caused by masking is reduced to a large extent.
Invariant Learning via Diffusion Dreamed Distribution Shifts
Using the validation set, we evaluate several popular DNN image classifiers and find that the classification performance of models generally suffers on our background diverse images.
Improved techniques for deterministic l2 robustness
In this work, we reduce this gap by introducing (a) a procedure to certify robustness of 1-Lipschitz CNNs by replacing the last linear layer with a 1-hidden layer MLP that significantly improves their performance for both standard and provably robust accuracy, (b) a method to significantly reduce the training time per epoch for Skew Orthogonal Convolution (SOC) layers (>30\% reduction for deeper networks) and (c) a class of pooling layers using the mathematical property that the $l_{2}$ distance of an input to a manifold is 1-Lipschitz.
Explicit Tradeoffs between Adversarial and Natural Distributional Robustness
In this setting, through theoretical and empirical analysis, we show that (i) adversarial training with $\ell_1$ and $\ell_2$ norms increases the model reliance on spurious features; (ii) For $\ell_\infty$ adversarial training, spurious reliance only occurs when the scale of the spurious features is larger than that of the core features; (iii) adversarial training can have an unintended consequence in reducing distributional robustness, specifically when spurious correlations are changed in the new test domain.
Goal-Conditioned Q-Learning as Knowledge Distillation
We empirically show that this can improve the performance of goal-conditioned off-policy reinforcement learning when the space of goals is high-dimensional.
Lethal Dose Conjecture on Data Poisoning
Deep Partition Aggregation (DPA) and its extension, Finite Aggregation (FA) are recent approaches for provable defenses against data poisoning, where they predict through the majority vote of many base models trained from different subsets of training set using a given learner.
Certifiably Robust Policy Learning against Adversarial Communication in Multi-agent Systems
Communication is important in many multi-agent reinforcement learning (MARL) problems for agents to share information and make good decisions.
Interpretable Mixture of Experts
In addition to constituting a standalone inherently-interpretable architecture, IME has the premise of being integrated with existing DNNs to offer interpretability to a subset of samples while maintaining the accuracy of the DNNs.
Core Risk Minimization using Salient ImageNet
Deep neural networks can be unreliable in the real world especially when they heavily use spurious features for their predictions.
Provable Adversarial Robustness for Fractional Lp Threat Models
Our approach builds on a recent work, Levine and Feizi (2021), which provides a provable defense against L_1 attacks.
Measuring Self-Supervised Representation Quality for Downstream Classification using Discriminative Features
Fine-tuning with Q-Score regularization can boost the linear probing accuracy of SSL models by up to 5. 8% on ImageNet-100 and 3. 7% on ImageNet-1K compared to their baselines.
Improved Certified Defenses against Data Poisoning with (Deterministic) Finite Aggregation
DPA predicts through an aggregation of base classifiers trained on disjoint subsets of data, thus restricting its sensitivity to dataset distortions.
Certifying Model Accuracy under Distribution Shifts
Certified robustness in machine learning has primarily focused on adversarial perturbations of the input with a fixed attack budget for each point in the data distribution.
A Comprehensive Study of Image Classification Model Sensitivity to Foregrounds, Backgrounds, and Visual Attributes
While datasets with single-label supervision have propelled rapid advances in image classification, additional annotations are necessary in order to quantitatively assess how models make predictions.
Interpolated Joint Space Adversarial Training for Robust and Generalizable Defenses
Under JSTM, we develop novel adversarial attacks and defenses.
Mutual Adversarial Training: Learning together is better than going alone
In other words, we can make a weak model more robust with the help of a strong teacher model.
Segment and Complete: Defending Object Detectors against Adversarial Patch Attacks with Robust Patch Detection
In addition, we design a robust shape completion algorithm, which is guaranteed to remove the entire patch from the images if the outputs of the patch segmenter are within a certain Hamming distance of the ground-truth patch masks.
Improving Deep Learning Interpretability by Saliency Guided Training
In this paper, we tackle this issue and introduce a {\it saliency guided training}procedure for neural networks to reduce noisy gradients used in predictions while retaining the predictive performance of the model.
On Hard Episodes in Meta-Learning
To address the issue of sub-par performance on hard episodes, we investigate and benchmark different meta-training strategies based on adversarial training and curriculum learning.
Salient ImageNet: How to discover spurious features in Deep Learning?
Our methodology is based on this key idea: to identify spurious or core \textit{visual features} used in model predictions, we identify spurious or core \textit{neural features} (penultimate layer neurons of a robust model) via limited human supervision (e. g., using top 5 activating images per feature).
FOCUS: Familiar Objects in Common and Uncommon Settings
Standard training datasets for deep learning often contain objects in common settings (e. g., "a horse on grass" or "a ship in water") since they are usually collected by randomly scraping the web.
Multi-Domain Self-Supervised Learning
In fact, training these methods on a combination of several domains often degrades the quality of learned representations compared to the models trained on a single domain.
Causal ImageNet: How to discover spurious features in Deep Learning?
Focusing on image classifications, we define causal attributes as the set of visual features that are always a part of the object while spurious attributes are the ones that are likely to {\it co-occur} with the object but not a part of it (e. g., attribute ``fingers" for class ``band aid").
Sample Efficient Detection and Classification of Adversarial Attacks via Self-Supervised Embeddings
In this paper, we propose a self-supervised method to detect adversarial attacks and classify them to their respective threat models, based on a linear model operating on the embeddings from a pre-trained self-supervised encoder.
Improved deterministic l2 robustness on CIFAR-10 and CIFAR-100
While $1$-Lipschitz CNNs can be designed by enforcing a $1$-Lipschitz constraint on each layer, training such networks requires each layer to have an orthogonal Jacobian matrix (for all inputs) to prevent the gradients from vanishing during backpropagation.
Policy Smoothing for Provably Robust Reinforcement Learning
Prior works in provable robustness in RL seek to certify the behaviour of the victim policy at every time-step against a non-adaptive adversary using methods developed for the static setting.
Skew Orthogonal Convolutions
Then, we use the Taylor series expansion of the Jacobian exponential to construct the SOC layer that is orthogonal.
Understanding Overparameterization in Generative Adversarial Networks
We also empirically study the role of model overparameterization in GANs using several large-scale experiments on CIFAR-10 and Celeb-A datasets.
Improved, Deterministic Smoothing for L_1 Certified Robustness
To the best of our knowledge, this is the first work to provide deterministic "randomized smoothing" for a norm-based adversarial threat model while allowing for an arbitrary classifier (i. e., a deep model) to be used as a base classifier and without requiring an exponential number of smoothing samples.
Low Curvature Activations Reduce Overfitting in Adversarial Training
In particular, we show that using activation functions with low (exact or approximate) curvature values has a regularization effect that significantly reduces both the standard and robust generalization gaps in adversarial training.
Deep Partition Aggregation: Provable Defenses against General Poisoning Attacks
Against general poisoning attacks where no prior certified defenses exists, DPA can certify $\geq$ 50% of test images against over 500 poison image insertions on MNIST, and nine insertions on CIFAR-10.
Understanding Over-parameterization in Generative Adversarial Networks
In this work, we present a comprehensive analysis of the importance of model over-parameterization in GANs both theoretically and empirically.
Perceptual Adversarial Robustness: Generalizable Defenses Against Unforeseen Threat Models
We call this threat model the neural perceptual threat model (NPTM); it includes adversarial examples with a bounded neural perceptual distance (a neural network-based approximation of the true perceptual distance) to natural images.
Fantastic Four: Differentiable and Efficient Bounds on Singular Values of Convolution Layers
Through experiments on MNIST and CIFAR-10, we demonstrate the effectiveness of our spectral bound in improving generalization and robustness of deep networks.
Benchmarking Deep Learning Interpretability in Time Series Predictions
Saliency methods are used extensively to highlight the importance of input features in model predictions.
Tight Second-Order Certificates for Randomized Smoothing
In this work, we show that there also exists a universal curvature-like bound for Gaussian random smoothing: given the exact value and gradient of a smoothed function, we compute a lower bound on the distance of a point to its closest adversarial example, called the Second-order Smoothing (SoS) robustness certificate.
Robust Optimal Transport with Applications in Generative Modeling and Domain Adaptation
To remedy this issue, robust formulations of OT with unbalanced marginal constraints have previously been proposed.
Winning Lottery Tickets in Deep Generative Models
In this paper, we confirm the existence of winning tickets in deep generative models such as GANs and VAEs.
GANs with Variational Entropy Regularizers: Applications in Mitigating the Mode-Collapse Issue
To tackle this issue, we take an information-theoretic approach and maximize a variational lower bound on the entropy of the generated samples to increase their diversity.
Certifying Confidence via Randomized Smoothing
It uses the probabilities of predicting the top two most-likely classes around an input point under a smoothing distribution to generate a certified radius for a classifier's prediction.
Dual Manifold Adversarial Robustness: Defense against Lp and non-Lp Adversarial Attacks
Using OM-ImageNet, we first show that adversarial training in the latent space of images improves both standard accuracy and robustness to on-manifold attacks.
Deep Partition Aggregation: Provable Defense against General Poisoning Attacks
Our defense against label-flipping attacks, SS-DPA, uses a semi-supervised learning algorithm as its base classifier model: each base classifier is trained using the entire unlabeled training set in addition to the labels for a partition.
Influence Functions in Deep Learning Are Fragile
Influence functions approximate the effect of training samples in test-time predictions and have a wide variety of applications in machine learning interpretability and uncertainty estimation.
Perceptual Adversarial Robustness: Defense Against Unseen Threat Models
We call this threat model the neural perceptual threat model (NPTM); it includes adversarial examples with a bounded neural perceptual distance (a neural network-based approximation of the true perceptual distance) to natural images.
Fairness Through Robustness: Investigating Robustness Disparity in Deep Learning
In this paper, we argue that traditional notions of fairness that are only based on models' outputs are not sufficient when the model is vulnerable to adversarial attacks.
Second-Order Provable Defenses against Adversarial Attacks
Second, we derive a computationally-efficient differentiable upper bound on the curvature of a deep network.
Unsupervised Anomaly Detection with Adversarial Mirrored AutoEncoders
Detecting out of distribution (OOD) samples is of paramount importance in all Machine Learning applications.
Out of Distribution (OOD) Detection
Representation Learning
+1
GANs with Conditional Independence Graphs: On Subadditivity of Probability Divergences
GANs, however, are designed in a model-free fashion where no additional information about the underlying distribution is available.
(De)Randomized Smoothing for Certifiable Defense against Patch Attacks
In this paper, we introduce a certifiable defense against patch attacks that guarantees for a given image and patch attack size, no patch adversarial examples exist.
Curse of Dimensionality on Randomized Smoothing for Certifiable Robustness
Notably, for $p \geq 2$, this dependence on $d$ is no better than that of the $\ell_p$-radius that can be certified using isotropic Gaussian smoothing, essentially putting a matching lower bound on the robustness radius.
Playing it Safe: Adversarial Robustness with an Abstain Option
We explore adversarial robustness in the setting in which it is acceptable for a classifier to abstain---that is, output no class---on adversarial examples.
Fantastic Four: Differentiable Bounds on Singular Values of Convolution Layers
Through experiments on MNIST and CIFAR-10, we demonstrate the effectiveness of our spectral bound in improving generalization and provable robustness of deep networks.
Robustness Certificates for Sparse Adversarial Attacks by Randomized Ablation
This is comparable to the observed empirical robustness of unprotected classifiers on MNIST to modern L_0 attacks, demonstrating the tightness of the proposed robustness certificate.
Adversarial Robustness of Flow-Based Generative Models
Finally, using a hybrid adversarial training procedure, we significantly boost the robustness of these generative models.
On Second-Order Group Influence Functions for Black-Box Predictions
Often we want to identify an influential group of training samples in a particular test prediction for a given machine learning model.
Quantum Wasserstein Generative Adversarial Networks
The study of quantum generative models is well-motivated, not only because of its importance in quantum machine learning and quantum chemistry but also because of the perspective of its implementation on near-term quantum machines.
Wasserstein Smoothing: Certified Robustness against Wasserstein Adversarial Attacks
An example of an attack method based on a non-additive threat model is the Wasserstein adversarial attack proposed by Wong et al. (2019), where the distance between an image and its adversarial example is determined by the Wasserstein metric ("earth-mover distance") between their normalized pixel intensities.
Deep k-NN Defense against Clean-label Data Poisoning Attacks
Targeted clean-label data poisoning is a type of adversarial attack on machine learning systems in which an adversary injects a few correctly-labeled, minimally-perturbed samples into the training data, causing a model to misclassify a particular test sample during inference.
Curvature-based Robustness Certificates against Adversarial Examples
We also use the curvature bound as a regularization term during the training of the network to boost its certified robustness against adversarial examples.
Interpretable Adversarial Training for Text
SPGD imposes a directional regularization constraint on input perturbations by projecting them onto the directions to nearby word embeddings with highest cosine similarities.
Functional Adversarial Attacks
For simplicity, we refer to functional adversarial attacks on image colors as ReColorAdv, which is the main focus of our experiments.
Certifiably Robust Interpretation in Deep Learning
Deep learning interpretation is essential to explain the reasoning behind model predictions.
Adversarially Robust Distillation
In addition to producing small models with high test accuracy like conventional distillation, ARD also passes the superior robustness of large networks onto the student.
Normalized Wasserstein Distance for Mixture Distributions with Applications in Adversarial Learning and Domain Adaptation
Using the proposed normalized Wasserstein measure leads to significant performance gains for mixture distributions with imbalanced mixture proportions compared to the vanilla Wasserstein distance.
Compressing GANs using Knowledge Distillation
From our experiments, we observe a qualitative limit for GAN's compression.
Robustness Certificates Against Adversarial Examples for ReLU Networks
These robustness certificates leverage the piece-wise linear structure of ReLU networks and use the fact that in a polyhedron around a given sample, the prediction function is linear.
Understanding Impacts of High-Order Loss Approximations and Features in Deep Learning Interpretation
Second, we compute the importance of group-features in deep learning interpretation by introducing a sparsity regularization term.
Porcupine Neural Networks: Approximating Neural Network Landscapes
Neural networks have been used prominently in several machine learning and statistics applications.
Entropic GANs meet VAEs: A Statistical Approach to Compute Sample Likelihoods in GANs
Building on the success of deep learning, two modern approaches to learn a probability model from the data are Generative Adversarial Networks (GANs) and Variational AutoEncoders (VAEs).
Are adversarial examples inevitable?
Using experiments, we explore the implications of theoretical guarantees for real-world problems and discuss how factors such as dimensionality and image complexity limit a classifier's robustness against adversarial examples.
Tensor Biclustering
Consider a dataset where data is collected on multiple features of multiple individuals over multiple times.
Understanding GANs: the LQG Setting
Generative Adversarial Networks (GANs) have become a popular method to learn a probability model from data.
Porcupine Neural Networks: (Almost) All Local Optima are Global
Neural networks have been used prominently in several machine learning and statistics applications.
Maximally Correlated Principal Component Analysis
For jointly Gaussian variables we show that the covariance matrix corresponding to the identity (or the negative of the identity) transformations majorizes covariance matrices of non-identity functions.
Network Maximal Correlation
For jointly Gaussian variables, we show that under some conditions the NMC optimization is an instance of the Max-Cut problem.
Maximum Likelihood Latent Space Embedding of Logistic Random Dot Product Graphs
A latent space model of particular interest is the Random Dot Product Graph (RDPG), which can be fit using an efficient spectral method; however, this method is based on a heuristic that can fail, even in simple cases.
