Search Results for author:
Found 57 papers, 25 papers with code
Foot-In-The-Door: A Multi-turn Jailbreak for LLMs
Ensuring AI safety is crucial as large language models become increasingly integrated into real-world applications.
SecureGaze: Defending Gaze Estimation Against Backdoor Attacks
In such attacks, adversaries inject backdoor triggers by poisoning the training data, creating a backdoor vulnerability: the model performs normally with benign inputs, but produces manipulated gaze directions when a specific trigger is present.
TrojanDec: Data-free Detection of Trojan Inputs in Self-supervised Learning
However, many studies showed that an attacker can embed a trojan into an encoder such that multiple downstream classifiers built based on the trojaned encoder simultaneously inherit the trojan behavior.
Data Free Backdoor Attacks
Existing backdoor attacks require either retraining the classifier with some clean data or modifying the model's architecture.
Stealing Training Graphs from Graph Neural Networks
As neural networks can memorize the training samples, the model parameters of GNNs have a high risk of leaking private training data.
Defending Deep Regression Models against Backdoor Attacks
Deep regression models are used in a wide variety of safety-critical applications, but are vulnerable to backdoor attacks.
PrivateGaze: Preserving User Privacy in Black-box Mobile Gaze Tracking Services
Eye gaze contains rich information about human attention and cognitive processes.
Certifiably Robust Image Watermark
In this work, we propose the first image watermarks with certified robustness guarantees against removal and forgery attacks.
Graph Neural Network Explanations are Fragile
Explainable Graph Neural Network (GNN) has emerged recently to foster the trust of using GNNs.
Towards General Robustness Verification of MaxPool-based Convolutional Neural Networks via Tightening Linear Approximation
The results show that MaxLin outperforms state-of-the-art tools with up to 110. 60% improvement regarding the certified lower bound and 5. 13 $\times$ speedup for the same neural networks.
ACE: A Model Poisoning Attack on Contribution Evaluation Methods in Federated Learning
Specifically, we show that any malicious client utilizing ACE could manipulate the parameters of its local model such that it is evaluated to have a high contribution by the server, even when its local training data is indeed of low quality.
MMCert: Provable Defense against Adversarial Attacks to Multi-modal Models
Moreover, we compare our MMCert with a state-of-the-art certified defense extended from unimodal models.
SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding
Our results show that SafeDecoding significantly reduces the attack success rate and harmfulness of jailbreak attacks without compromising the helpfulness of responses to benign user queries.
PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models
Based on this attack surface, we propose PoisonedRAG, the first knowledge corruption attack to RAG, where an attacker could inject a few malicious texts into the knowledge database of a RAG system to induce an LLM to generate an attacker-chosen target answer for an attacker-chosen target question.
Brave: Byzantine-Resilient and Privacy-Preserving Peer-to-Peer Federated Learning
Our results show that the global model learned with Brave in the presence of adversaries achieves comparable classification accuracy to a global model trained in the absence of any adversary.
Data Poisoning based Backdoor Attacks to Contrastive Learning
In this work we take the first step to analyze the limitations of existing backdoor attacks and propose new DPBAs called CorruptEncoder to CL.
TextGuard: Provable Defense against Backdoor Attacks on Text Classification
In this work, we propose TextGuard, the first provable defense against backdoor attacks on text classification.
Identifying and Mitigating Vulnerabilities in LLM-Integrated Applications
Successful exploits of the identified vulnerabilities result in the users receiving responses tailored to the intent of a threat initiator.
IMPRESS: Evaluating the Resilience of Imperceptible Perturbations Against Unauthorized Data Usage in Diffusion-Based Generative AI
IMPRESS is based on the key observation that imperceptible perturbations could lead to a perceptible inconsistency between the original image and the diffusion-reconstructed image, which can be used to devise a new optimization strategy for purifying the image, which may weaken the protection of the original image from unauthorized data usage (e. g., style mimicking, malicious editing).
Formalizing and Benchmarking Prompt Injection Attacks and Defenses
Existing attacks are special cases in our framework.
On the Safety of Open-Sourced Large Language Models: Does Alignment Really Prevent Them From Being Misused?
A natural question is "could alignment really prevent those open-sourced large language models from being misused to generate undesired content?''.
PORE: Provably Robust Recommender Systems against Data Poisoning Attacks
PORE can transform any existing recommender system to be provably robust against any untargeted data poisoning attacks, which aim to reduce the overall performance of a recommender system.
PointCert: Point Cloud Classification with Deterministic Certified Robustness Guarantees
Existing certified defenses against adversarial point clouds suffer from a key limitation: their certified robustness guarantees are probabilistic, i. e., they produce an incorrect certified robustness guarantee with some probability.
REaaS: Enabling Adversarially Robust Downstream Classifiers via Robust Encoder as a Service
For the first question, we show that the cloud service only needs to provide two APIs, which we carefully design, to enable a client to certify the robustness of its downstream classifier with a minimal number of queries to the APIs.
Pre-trained Encoders in Self-Supervised Learning Improve Secure and Privacy-preserving Supervised Learning
In this work, we perform the first systematic, principled measurement study to understand whether and when a pre-trained encoder can address the limitations of secure or privacy-preserving supervised learning algorithms.
CorruptEncoder: Data Poisoning based Backdoor Attacks to Contrastive Learning
In this work, we take the first step to analyze the limitations of existing backdoor attacks and propose new DPBAs called CorruptEncoder to CL.
FedRecover: Recovering from Poisoning Attacks in Federated Learning using Historical Information
Existing defenses focus on preventing a small number of malicious clients from poisoning the global model via robust federated learning methods and detecting malicious clients when there are a large number of them.
MultiGuard: Provably Robust Multi-label Classification against Adversarial Examples
In this work, we propose MultiGuard, the first provably robust defense against adversarial examples to multi-label classification.
FLCert: Provably Secure Federated Learning against Poisoning Attacks
Our key idea is to divide the clients into groups, learn a global model for each group of clients using any existing federated learning method, and take a majority vote among the global models to classify a test input.
FLDetector: Defending Federated Learning Against Model Poisoning Attacks via Detecting Malicious Clients
FLDetector aims to detect and remove the majority of the malicious clients such that a Byzantine-robust FL method can learn an accurate global model using the remaining clients.
PoisonedEncoder: Poisoning the Unlabeled Pre-training Data in Contrastive Learning
In this work, we propose PoisonedEncoder, a data poisoning attack to contrastive learning.
StolenEncoder: Stealing Pre-trained Encoders in Self-supervised Learning
A pre-trained encoder may be deemed confidential because its training requires lots of data and computation resources as well as its public release may facilitate misuse of AI, e. g., for deepfakes generation.
10 Security and Privacy Problems in Large Foundation Models
A pre-trained foundation model is like an ``operating system'' of the AI ecosystem.
Anomaly Detection In Surveillance Videos
Self-Supervised Learning
EncoderMI: Membership Inference against Pre-trained Encoders in Contrastive Learning
EncoderMI can be used 1) by a data owner to audit whether its (public) data was used to pre-train an image encoder without its authorization or 2) by an attacker to compromise privacy of the training data when it is private/sensitive.
BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learning
In particular, our BadEncoder injects backdoors into a pre-trained image encoder such that the downstream classifiers built based on the backdoored image encoder for different downstream tasks simultaneously inherit the backdoor behavior.
PointGuard: Provably Robust 3D Point Cloud Classification
Our first major theoretical contribution is that we show PointGuard provably predicts the same label for a 3D point cloud when the number of adversarially modified, added, and/or deleted points is bounded.
Provably Secure Federated Learning against Malicious Clients
We show that our ensemble federated learning with any base federated learning algorithm is provably secure against malicious clients.
Semi-Supervised Node Classification on Graphs: Markov Random Fields vs. Graph Neural Networks
In this work, we aim to address the key limitation of existing pMRF-based methods.
Certified Robustness of Nearest Neighbors against Data Poisoning and Backdoor Attacks
Moreover, our evaluation results on MNIST and CIFAR10 show that the intrinsic certified robustness guarantees of kNN and rNN outperform those provided by state-of-the-art certified defenses.
Almost Tight L0-norm Certified Robustness of Top-k Predictions against Adversarial Perturbations
For instance, our method can build a classifier that achieves a certified top-3 accuracy of 69. 2\% on ImageNet when an attacker can arbitrarily perturb 5 pixels of a testing image.
Robust and Verifiable Information Embedding Attacks to Deep Neural Networks via Error-Correcting Codes
Moreover, to be robust against post-processing, we leverage Turbo codes, a type of error-correcting codes, to encode the message before embedding it to the DNN classifier.
Certified Robustness of Graph Neural Networks against Adversarial Structural Perturbation
Specifically, we prove the certified robustness guarantee of any GNN for both node and graph classifications against structural perturbation.
Cryptography and Security
On the Intrinsic Differential Privacy of Bagging
Bagging, a popular ensemble learning framework, randomly creates some subsamples of the training data, trains a base model for each subsample using a base learner, and takes majority vote among the base models when making predictions.
Intrinsic Certified Robustness of Bagging against Data Poisoning Attacks
Specifically, we show that bagging with an arbitrary base learning algorithm provably predicts the same label for a testing example when the number of modified, deleted, and/or inserted training examples is bounded by a threshold.
Backdoor Attacks to Graph Neural Networks
Specifically, we propose a \emph{subgraph based backdoor attack} to GNN for graph classification.
Stealing Links from Graph Neural Networks
In this work, we propose the first attacks to steal a graph from the outputs of a GNN model that is trained on the graph.
On Certifying Robustness against Backdoor Attacks via Randomized Smoothing
Specifically, in this work, we study the feasibility and effectiveness of certifying robustness against backdoor attacks using a recent technique called randomized smoothing.
Certified Robustness of Community Detection against Adversarial Structural Perturbation via Randomized Smoothing
However, several recent studies showed that community detection is vulnerable to adversarial structural perturbation.
Certified Robustness for Top-k Predictions against Adversarial Perturbations via Randomized Smoothing
For example, our method can obtain an ImageNet classifier with a certified top-5 accuracy of 62. 8\% when the $\ell_2$-norms of the adversarial perturbations are less than 0. 5 (=127/255).
Local Model Poisoning Attacks to Byzantine-Robust Federated Learning
Our empirical results on four real-world datasets show that our attacks can substantially increase the error rates of the models learnt by the federated learning methods that were claimed to be robust against Byzantine failures of some client devices.
Data Poisoning Attacks to Local Differential Privacy Protocols
Local Differential Privacy (LDP) protocols enable an untrusted data collector to perform privacy-preserving data analytics.
Data Poisoning
Cryptography and Security
Distributed, Parallel, and Cluster Computing
IPGuard: Protecting Intellectual Property of Deep Neural Networks via Fingerprinting the Classification Boundary
Our key observation is that a DNN classifier can be uniquely represented by its classification boundary.
MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examples
Specifically, given a black-box access to the target classifier, the attacker trains a binary classifier, which takes a data sample's confidence score vector predicted by the target classifier as an input and predicts the data sample to be a member or non-member of the target classifier's training dataset.
Defending against Machine Learning based Inference Attacks via Adversarial Examples: Opportunities and Challenges
To defend against inference attacks, we can add carefully crafted noise into the public data to turn them into adversarial examples, such that attackers' classifiers make incorrect predictions for the private data.
Graph-based Security and Privacy Analytics via Collective Classification with Joint Weight Learning and Propagation
To address the computational challenge, we propose to jointly learn the edge weights and propagate the reputation scores, which is essentially an approximate solution to the optimization problem.
AttriGuard: A Practical Defense Against Attribute Inference Attacks via Adversarial Machine Learning
Specifically, game-theoretic defenses require solving intractable optimization problems, while correlation-based defenses incur large utility loss of users' public data.
Object Proposal by Multi-Branch Hierarchical Segmentation
Hierarchical segmentation based object proposal methods have become an important step in modern object detection paradigm.
