Search Results for author:
Found 107 papers, 56 papers with code
Exploring the Vulnerability of Natural Language Processing Models via Universal Adversarial Texts
Universal adversarial texts (UATs) refer to short pieces of text units that can largely affect the predictions of NLP models.
X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIP
As Contrastive Language-Image Pre-training (CLIP) models are increasingly adopted for diverse downstream tasks and integrated into large vision-language models (VLMs), their susceptibility to adversarial perturbations has emerged as a critical concern.
A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment
Currently, existing surveys on LLM safety primarily focus on specific stages of the LLM lifecycle, e. g., deployment phase or fine-tuning phase, lacking a comprehensive understanding of the entire "lifechain" of LLMs.
OmniSVG: A Unified Scalable Vector Graphics Generation Model
Scalable Vector Graphics (SVG) is an important image format widely adopted in graphic design because of their resolution independence and editability.
Identity Lock: Locking API Fine-tuned LLMs With Identity-based Wake Words
To implement this, we propose a fine-tuning method named IdentityLock that integrates the wake words at the beginning of a large proportion (90%) of the training text prompts, while modifying the responses of the remaining 10% to indicate refusals.
Reinforced Diffuser for Red Teaming Large Vision-Language Models
The rapid advancement of large Vision-Language Models (VLMs) has raised significant safety concerns, particularly regarding their vulnerability to jailbreak attacks.
Detecting Backdoor Samples in Contrastive Language Image Pretraining
Contrastive language-image pretraining (CLIP) has been found to be vulnerable to poisoning backdoor attacks where the adversary can achieve an almost perfect attack success rate on CLIP models by poisoning only 0. 01\% of the training dataset.
Safety at Scale: A Comprehensive Survey of Large Model Safety
The rapid advancement of large models, driven by their exceptional abilities in learning and generalization through large-scale pre-training, has reshaped the landscape of Artificial Intelligence (AI).
CALM: Curiosity-Driven Auditing for Large Language Models
We treat this type of auditing as a black-box optimization problem where the goal is to automatically uncover input-output pairs of the target LLMs that exhibit illegal, immoral, or unsafe behaviors.
AIM: Additional Image Guided Generation of Transferable Adversarial Attacks
In this work, we focus on generative approaches for targeted transferable attacks.
Free-Form Motion Control: A Synthetic Video Generation Dataset with Controllable Camera and Object Motions
Controlling the movements of dynamic objects and the camera within generated videos is a meaningful yet challenging task.
HoneypotNet: Backdoor Attacks Against Model Extraction
In this work, we introduce a new defense paradigm called attack as defense which modifies the model's output to be poisonous such that any malicious users that attempt to use the output to train a substitute model will be poisoned.
DiffPatch: Generating Customizable Adversarial Patches using Diffusion Model
However, most existing adversarial patch generation methods prioritize attack effectiveness over stealthiness, resulting in patches that are aesthetically unpleasing.
Adversarial Prompt Distillation for Vision-Language Models
In this work, we propose a novel method called Adversarial Prompt Distillation (APD) that combines APT with knowledge distillation to boost the adversarial robustness of CLIP.
TAPT: Test-Time Adversarial Prompt Tuning for Robust Inference in Vision-Language Models
TAPT is a test-time defense method that learns defensive bimodal (textual and visual) prompts to robustify the inference process of CLIP.
Towards Million-Scale Adversarial Robustness Evaluation With Stronger Individual Attacks
Over the past decade, a large number of white-box adversarial robustness evaluation methods (i. e., attacks) have been proposed, ranging from single-step to multi-step methods and from individual to ensemble methods.
IDEATOR: Jailbreaking Large Vision-Language Models Using Themselves
Specifically, IDEATOR uses a VLM to create targeted jailbreak texts and pairs them with jailbreak images generated by a state-of-the-art diffusion model.
BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attacks
In this work, we focus on black-box defense for VLMs against jailbreak attacks.
Expose Before You Defend: Unifying and Enhancing Backdoor Defenses via Exposed Models
Specifically, EBYD first exposes the backdoor functionality in the backdoored model through a model preprocessing step called backdoor exposure, and then applies detection and removal methods to the exposed model to identify and eliminate the backdoor features.
UnSeg: One Universal Unlearnable Example Generator is Enough against All Image Segmentation
In this work, we exploit the concept of unlearnable examples to make images unusable to model training by generating and adding unlearnable noise into the original images.
On the Adversarial Transferability of Generalized "Skip Connections"
Specifically, in ResNet-like models (with skip connections), we find that using more gradients from the skip connections rather than the residual modules according to a decay factor during backpropagation allows one to craft adversarial examples with high transferability.
AnyAttack: Towards Large-scale Self-supervised Adversarial Attacks on Vision-language Models
Due to their multimodal capabilities, Vision-Language Models (VLMs) have found numerous impactful applications in real-world scenarios.
Extracting Training Data from Unconditional Diffusion Models
As diffusion probabilistic models (DPMs) are being employed as mainstream models for Generative Artificial Intelligence (GenAI), the study of their memorization has attracted growing attention.
Federated Instruction Tuning of LLMs with Domain Coverage Augmentation
Our experiments reveal that the cross-client domain coverage, rather than data heterogeneity, drives model performance in FedDIT.
BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks on Large Language Models
Generative Large Language Models (LLMs) have made significant strides across various tasks, but they remain vulnerable to backdoor attacks, where specific triggers in the prompt cause the LLM to generate adversary-desired responses.
EnJa: Ensemble Jailbreak on Large Language Models
As Large Language Models (LLMs) are increasingly being deployed in safety-critical applications, their vulnerability to potential jailbreaks -- malicious prompts that can disable the safety mechanism of LLMs -- has attracted growing research attention.
AdvQDet: Detecting Query-Based Adversarial Attacks with Adversarial Contrastive Prompt Tuning
During this process, the queries made to the target model are intermediate adversarial examples crafted at the previous attack step, which share high similarities in the pixel space.
Downstream Transfer Attack: Adversarial Attacks on Downstream Models with Pre-trained Vision Transformers
With the advancement of vision transformers (ViTs) and self-supervised learning (SSL) techniques, pre-trained large ViTs have become the new foundation models for computer vision applications.
Constrained Intrinsic Motivation for Reinforcement Learning
To tackle these problems, we propose \emph{Constrained Intrinsic Motivation (CIM)} for RFPT and EIM tasks, respectively: 1) CIM for RFPT maximizes the lower bound of the conditional state entropy subject to an alignment constraint on the state encoder network for efficient dynamic and diverse skill discovery and state coverage maximization; 2) CIM for EIM leverages constrained policy optimization to adaptively adjust the coefficient of the intrinsic objective to mitigate the distraction from the intrinsic objective.
CHASE: A Causal Heterogeneous Graph based Framework for Root Cause Analysis in Multimodal Microservice Systems
In recent years, the widespread adoption of distributed microservice architectures within the industry has significantly increased the demand for enhanced system availability and robustness.
A Survey of Multimodal-Guided Image Editing with Text-to-Image Diffusion Models
Image editing aims to edit the given synthetic or real image to meet the specific requirements from users.
Extracting Training Data from Unconditional Diffusion Models
In this work, we aim to establish a theoretical understanding of memorization in DPMs with 1) a memorization metric for theoretical analysis, 2) an analysis of conditional memorization with informative and random labels, and 3) two better evaluation metrics for measuring memorization.
White-box Multimodal Jailbreaks Against Large Vision-Language Models
Subsequently, an adversarial text suffix is integrated and co-optimized with the adversarial image prefix to maximize the probability of eliciting affirmative responses to various harmful instructions.
ModelLock: Locking Your Model With a Spell
This paper presents a novel model protection paradigm ModelLock that locks (destroys) the performance of a model on normal clean data so as to make it unusable or unextractable without the right key.
FedCAda: Adaptive Client-Side Optimization for Accelerated and Stable Federated Learning
This comparative analysis revealed that due to the limited information contained within client models from other clients during the initial stages of federated learning, more substantial constraints need to be imposed on the parameters of the adaptive algorithm.
Special Characters Attack: Toward Scalable Training Data Extraction From Large Language Models
In this paper, we take a step further and show that certain special characters or their combinations with English letters are stronger memory triggers, leading to more severe data leakage.
FedEGG: Federated Learning with Explicit Global Guidance
Federated Learning (FL) holds great potential for diverse applications owing to its privacy-preserving nature.
The Double-Edged Sword of Input Perturbations to Robust Accurate Fairness
Deep neural networks (DNNs) are known to be sensitive to adversarial input perturbations, leading to a reduction in either prediction accuracy or individual fairness.
Whose Side Are You On? Investigating the Political Stance of Large Language Models
Large Language Models (LLMs) have gained significant popularity for their application in various everyday tasks such as text generation, summarization, and information retrieval.
TokenMark: A Modality-Agnostic Watermark for Pre-trained Transformers
Watermarking is a critical tool for model ownership verification.
Unlearnable Examples For Time Series
In this work, we introduce the first UE generation method to protect time series data from unauthorized training by deep learning models.
Shortcuts Everywhere and Nowhere: Exploring Multi-Trigger Backdoor Attacks
Backdoor attacks have become a significant threat to the pre-training and deployment of deep neural networks (DNNs).
LDReg: Local Dimensionality Regularized Self-Supervised Learning
Representations learned via self-supervised learning (SSL) can be susceptible to dimensional collapse, where the learned representation subspace is of extremely low dimensionality and thus fails to represent the full data distribution and modalities.
End-to-End Anti-Backdoor Learning on Images and Time Series
Backdoor attacks present a substantial security concern for deep learning models, especially those utilized in applications critical to safety and security.
Adversarial Prompt Tuning for Vision-Language Models
With the rapid advancement of multimodal learning, pre-trained Vision-Language Models (VLMs) such as CLIP have demonstrated remarkable capacities in bridging the gap between visual and language modalities.
Fake Alignment: Are LLMs Really Aligned Well?
The growing awareness of safety concerns in large language models (LLMs) has sparked considerable interest in the evaluation of safety.
Fuse Your Latents: Video Editing with Multi-source Latent Diffusion Models
This paper is the first to reveal that T2I and T2V LDMs can complement each other in terms of structure and temporal consistency, ultimately generating high-quality videos.
On the Importance of Spatial Relations for Few-shot Action Recognition
We are thus motivated to investigate the importance of spatial relations and propose a more accurate few-shot action recognition method that leverages both spatial and temporal information.
Learning from Heterogeneity: A Dynamic Learning Framework for Hypergraphs
Graph neural network (GNN) has gained increasing popularity in recent years owing to its capability and flexibility in modeling complex graph structure data.
Reconstructive Neuron Pruning for Backdoor Defense
Specifically, RNP first unlearns the neurons by maximizing the model's error on a small subset of clean samples and then recovers the neurons by minimizing the model's error on the same data.
Toward Evaluating Robustness of Reinforcement Learning with Adversarial Policy
Our experiments validate the effectiveness of the four types of adversarial intrinsic regularizers and the bias-reduction method in enhancing black-box adversarial policy learning across a variety of environments.
Distilling Cognitive Backdoor Patterns within an Image
We conduct extensive experiments to show that CD can robustly detect a wide range of advanced backdoor attacks.
Unlearnable Clusters: Towards Label-agnostic Unlearnable Examples
Furthermore, we propose to leverage VisionandLanguage Pre-trained Models (VLPMs) like CLIP as the surrogate model to improve the transferability of the crafted UCs to diverse domains.
CIM: Constrained Intrinsic Motivation for Sparse-Reward Continuous Control
Intrinsic motivation is a promising exploration technique for solving reinforcement learning tasks with sparse or absent extrinsic rewards.
Backdoor Attacks on Time Series: A Generative Approach
We find that, compared to images, it can be more challenging to achieve the two goals on time series.
Transferable Unlearnable Examples
The unlearnable strategies have been introduced to prevent third parties from training on the data without permission.
Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models
In this work, we take the first step to exploit the pre-trained (unfine-tuned) weights to mitigate backdoors in fine-tuned language models.
Ranked #1 on
Sentiment Analysis
on SST-2 Binary classification
(Attack Success Rate metric)
Backdoor Attacks on Crowd Counting
In this paper, we propose two novel Density Manipulation Backdoor Attacks (DMBA$^{-}$ and DMBA$^{+}$) to attack the model to produce arbitrarily large or small density estimations.
CalFAT: Calibrated Federated Adversarial Training with Label Skewness
In this paper, we study the problem of FAT under label skewness, and reveal one root cause of the training instability and natural accuracy degradation issues: skewed labels lead to non-identical class probabilities and heterogeneous local models.
VeriFi: Towards Verifiable Federated Unlearning
One desirable property for FL is the implementation of the right to be forgotten (RTBF), i. e., a leaving participant has the right to request to delete its private data from the global model.
Few-Shot Backdoor Attacks on Visual Object Tracking
Visual object tracking (VOT) has been widely adopted in mission-critical applications, such as autonomous driving and intelligent surveillance systems.
On the Convergence and Robustness of Adversarial Training
In this paper, we propose such a criterion, namely First-Order Stationary Condition for constrained optimization (FOSC), to quantitatively evaluate the convergence quality of adversarial examples found in the inner maximization.
Gradient Driven Rewards to Guarantee Fairness in Collaborative Machine Learning
In this paper, we adopt federated learning as a gradient-based formalization of collaborative machine learning, propose a novel cosine gradient Shapley value to evaluate the agents’ uploaded model parameter updates/gradients, and design theoretically guaranteed fair rewards in the form of better model performance.
SpineOne: A One-Stage Detection Framework for Degenerative Discs and Vertebrae
Spinal degeneration plagues many elders, office workers, and even the younger generations.
Alpha-IoU: A Family of Power Intersection over Union Losses for Bounding Box Regression
Bounding box (bbox) regression is a fundamental task in computer vision.
Anti-Backdoor Learning: Training Clean Models on Poisoned Data
From this view, we identify two inherent characteristics of backdoor attacks as their weaknesses: 1) the models learn backdoored data much faster than learning with clean data, and the stronger the attack the faster the model converges on backdoored data; 2) the backdoor task is tied to a specific class (the backdoor target class).
ECG-ATK-GAN: Robustness against Adversarial Attacks on ECGs using Conditional Generative Adversarial Networks
The experiment confirms that our model is more robust against such adversarial attacks for classifying arrhythmia with high accuracy.
Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks
Specifically, we make the following key observations: 1) more parameters (higher model capacity) does not necessarily help adversarial robustness; 2) reducing capacity at the last stage (the last group of blocks) of the network can actually improve adversarial robustness; and 3) under the same parameter budget, there exists an optimal architectural configuration for adversarial robustness.
Understanding Graph Learning with Local Intrinsic Dimensionality
Many real-world problems can be formulated as graphs and solved by graph learning techniques.
FedDiscrete: A Secure Federated Learning Algorithm Against Weight Poisoning
Federated learning (FL) is a privacy-aware collaborative learning paradigm that allows multiple parties to jointly train a machine learning model without sharing their private data.
Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Better
We empirically demonstrate the effectiveness of our RSLAD approach over existing adversarial training and distillation methods in improving the robustness of small models against state-of-the-art attacks including the AutoAttack.
ECG-Adv-GAN: Detecting ECG Adversarial Examples with Conditional Generative Adversarial Networks
Moreover, the model is conditioned on class-specific ECG signals to synthesize realistic adversarial examples.
Adversarial Interaction Attacks: Fooling AI to Misinterpret Human Intentions
Understanding the actions of both humans and artificial intelligence (AI) agents is important before modern AI systems can be fully integrated into our daily life.
Noise Doesn't Lie: Towards Universal Detection of Deep Inpainting
A promising countermeasure against such forgeries is deep inpainting detection, which aims to locate the inpainted regions in an image.
$\alpha$-IoU: A Family of Power Intersection over Union Losses for Bounding Box Regression
Bounding box (bbox) regression is a fundamental task in computer vision.
Dual Head Adversarial Training
Deep neural networks (DNNs) are known to be vulnerable to adversarial examples/attacks, raising concerns about their reliability in safety-critical applications.
Improving Adversarial Robustness via Channel-wise Activation Suppressing
The study of adversarial examples and their activation has attracted significant attention for secure and robust learning with deep neural networks (DNNs).
Anomaly Detection for Scenario-based Insider Activities using CGAN Augmented Data
Insider threats are the cyber attacks from within the trusted entities of an organization.
RobOT: Robustness-Oriented Testing for Deep Learning Systems
A key part of RobOT is a quantitative measurement on 1) the value of each test case in improving model robustness (often via retraining), and 2) the convergence quality of the model robustness improvement.
Software Engineering
What Do Deep Nets Learn? Class-wise Patterns Revealed in the Input Space
In this paper, we focus on image classification and propose a method to visualize and understand the class-wise knowledge (patterns) learned by DNNs under three different settings including natural, backdoor and adversarial.
Adversarial Interaction Attack: Fooling AI to Misinterpret Human Intentions
Understanding the actions of both humans and artificial intelligence (AI) agents is important before modern AI systems can be fully integrated into our daily life.
Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks
NAD utilizes a teacher network to guide the finetuning of the backdoored student network on a small clean subset of data such that the intermediate-layer attention of the student network aligns with that of the teacher network.
Unlearnable Examples: Making Personal Data Unexploitable
This paper raises the question: \emph{can data be made unlearnable for deep learning models?}
WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection
WildDeepfake is a small dataset that can be used, in addition to existing datasets, to develop and test the effectiveness of deepfake detectors against real-world deepfakes.
Neural Architecture Search via Combinatorial Multi-Armed Bandit
NAS can be performed via policy gradient, evolutionary algorithms, differentiable architecture search or tree-search methods.
Privacy and Robustness in Federated Learning: Attacks and Defenses
Besides training powerful global models, it is of paramount importance to design FL systems that have privacy guarantees and are resistant to different types of adversaries.
Imbalanced Gradients: A New Cause of Overestimated Adversarial Robustness
Evaluating the robustness of a defense model is a challenging task in adversarial robustness research.
Short-Term and Long-Term Context Aggregation Network for Video Inpainting
Video inpainting aims to restore missing regions of a video and has many applications such as video editing and object removal.
How to Democratise and Protect AI: Fair and Differentially Private Decentralised Deep Learning
This paper firstly considers the research problem of fairness in collaborative deep learning, while ensuring privacy.
Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks
A backdoor attack installs a backdoor into the victim model by injecting a backdoor pattern into a small proportion of the training data.
Imbalanced Gradients: A Subtle Cause of Overestimated Adversarial Robustness
Evaluating the robustness of a defense model is a challenging task in adversarial robustness research.
Normalized Loss Functions for Deep Learning with Noisy Labels
However, in practice, simply being robust is not sufficient for a loss function to train accurate DNNs.
Ranked #32 on
Image Classification
on mini WebVision 1.0
(ImageNet Top-1 Accuracy metric)
Improving Adversarial Robustness Requires Revisiting Misclassified Examples
In this paper, we investigate the distinctive influence of misclassified and correctly classified examples on the final robustness of adversarial training.
Adversarial Camouflage: Hiding Physical-World Attacks with Natural Styles
Deep neural networks (DNNs) are known to be vulnerable to adversarial examples.
Clean-Label Backdoor Attacks on Video Recognition Models
We propose the use of a universal adversarial trigger as the backdoor trigger to attack video recognition models, a situation where backdoor attacks are likely to be challenged by the above 4 strict conditions.
Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets
We find that using more gradients from the skip connections rather than the residual modules according to a decay factor, allows one to craft adversarial examples with high transferability.
Symmetric Cross Entropy for Robust Learning with Noisy Labels
In this paper, we show that DNN learning with Cross Entropy (CE) exhibits overfitting to noisy labels on some classes ("easy" classes), but more surprisingly, it also suffers from significant under learning on some other classes ("hard" classes).
Ranked #44 on
Image Classification
on Clothing1M
Generative Image Inpainting with Submanifold Alignment
Image inpainting aims at restoring missing regions of corrupted images, which has many applications such as image restoration and object removal.
Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systems
This raises safety concerns about the deployment of these systems in clinical settings.
Towards Fair and Privacy-Preserving Federated Deep Models
This problem can be addressed by either a centralized framework that deploys a central server to train a global model on the joint data from all parties, or a distributed framework that leverages a parameter server to aggregate local model updates.
Quality Evaluation of GANs Using Cross Local Intrinsic Dimensionality
In this paper, we demonstrate that an intrinsic dimensional characterization of the data space learned by a GAN model leads to an effective evaluation metric for GAN quality.
Black-box Adversarial Attacks on Video Recognition Models
Using three benchmark video datasets, we demonstrate that V-BAD can craft both untargeted and targeted attacks to fool two state-of-the-art deep video recognition models.
Dimensionality-Driven Learning with Noisy Labels
Datasets with significant proportions of noisy (incorrect) class labels present challenges for training accurate Deep Neural Networks (DNNs).
Ranked #41 on
Image Classification
on mini WebVision 1.0
Iterative Learning with Open-set Noisy Labels
We refer to this more complex scenario as the \textbf{open-set noisy label} problem and show that it is nontrivial in order to make accurate predictions.
Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality
Deep Neural Networks (DNNs) have recently been shown to be vulnerable against adversarial examples, which are carefully crafted instances that can mislead DNNs to make errors during prediction.
Providing Effective Real-time Feedback in Simulation-based Surgical Training
Experimental results in a temporal bone surgery simulation show that the proposed method is able to extract highly effective feedback at a high level of efficiency.
Adversarial Generation of Real-time Feedback with Neural Networks for Simulation-based Training
It is the aim of this paper to develop an efficient and effective feedback generation method for the provision of real-time feedback in SBT.
