Search Results for author:
Found 92 papers, 52 papers with code
RealSafe-R1: Safety-Aligned DeepSeek-R1 without Compromising Reasoning Capability
Both quantitative experiments and qualitative case studies demonstrate the models' improvements, which are shown in their safety guardrails against both harmful queries and jailbreak attacks.
STAIR: Improving Safety Alignment with Introspective Reasoning
Ensuring the safety and harmlessness of Large Language Models (LLMs) has become equally critical as their performance in applications.
Safety at Scale: A Comprehensive Survey of Large Model Safety
The rapid advancement of large models, driven by their exceptional abilities in learning and generalization through large-scale pre-training, has reshaped the landscape of Artificial Intelligence (AI).
Towards the Worst-case Robustness of Large Language Models
We propose a general tight lower bound for randomized smoothing using fractional knapsack solvers or 0-1 knapsack solvers, and using them to bound the worst-case robustness of all stochastic defenses.
Libra-Leaderboard: Towards Responsible AI through a Balanced Leaderboard of Safety and Capability
To address this gap, we introduce Libra-Leaderboard, a comprehensive framework designed to rank LLMs through a balanced evaluation of performance and safety.
AdvDreamer Unveils: Are Vision-Language Models Truly Ready for Real-World 3D Variations?
To systematically evaluate VLMs' robustness to real-world 3D variations, we propose AdvDreamer, the first framework that generates physically reproducible adversarial 3D transformation (Adv-3DT) samples from single-view images.
Exploring Aleatoric Uncertainty in Object Detection via Vision Foundation Models
Datasets collected from the open world unavoidably suffer from various forms of randomness or noiseness, leading to the ubiquity of aleatoric (data) uncertainty.
Scaling Laws for Black box Adversarial Attacks
By analyzing the relationship between the number of surrogate models and transferability of adversarial examples, we conclude with clear scaling laws, emphasizing the potential of using more surrogate models to enhance adversarial transferability.
Real-world Adversarial Defense against Patch Attacks based on Diffusion Model
Adversarial patches present significant challenges to the robustness of deep learning models, making the development of effective defenses become critical for real-world applications.
T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Models
The recent development of Sora leads to a new era in text-to-video (T2V) generation.
MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Models
Despite the superior capabilities of Multimodal Large Language Models (MLLMs) across diverse tasks, they still face significant trustworthiness challenges.
AutoBreach: Universal and Adaptive Jailbreaking with Efficient Wordplay-Guided Optimization
Despite the widespread application of large language models (LLMs) across various tasks, recent studies indicate that they are susceptible to jailbreak attacks, which can render their defense mechanisms ineffective.
Efficient Black-box Adversarial Attacks via Bayesian Optimization Guided by a Function Prior
In this paper, we propose a Prior-guided Bayesian Optimization (P-BO) algorithm that leverages the surrogate model as a global function prior in black-box adversarial attacks.
Membership Inference on Text-to-Image Diffusion Models via Conditional Likelihood Discrepancy
Text-to-image diffusion models have achieved tremendous success in the field of controllable image generation, while also coming along with issues of privacy leakage and data copyrights.
The RoboDrive Challenge: Drive Anytime Anywhere in Any Condition
In the realm of autonomous driving, robust perception under out-of-distribution conditions is paramount for the safe deployment of vehicles.
Omniview-Tuning: Boosting Viewpoint Invariance of Vision-Language Pre-training Models
Vision-Language Pre-training (VLP) models like CLIP have achieved remarkable success in computer vision and particularly demonstrated superior robustness to distribution shifts of 2D images.
Exploring the Transferability of Visual Prompting for Multimodal Large Language Models
To achieve this, we propose Transferable Visual Prompting (TVP), a simple and effective approach to generate visual prompts that can transfer to different models and improve their performance on downstream tasks after trained on only one model.
FaceCat: Enhancing Face Recognition Security with a Unified Diffusion Model
Face anti-spoofing (FAS) and adversarial detection (FAD) have been regarded as critical technologies to ensure the safety of face recognition systems.
Embodied Active Defense: Leveraging Recurrent Feedback to Counter Adversarial Patches
The vulnerability of deep neural networks to adversarial patches has motivated numerous defense strategies for boosting model robustness.
Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction
We evaluate DRA across various open-source and closed-source models, showcasing state-of-the-art jailbreak success rates and attack efficiency.
BSPA: Exploring Black-box Stealthy Prompt Attacks against Image Generators
It allows users to design specific prompts to generate realistic images through some black-box APIs.
Diffusion Models are Certifiably Robust Classifiers
Experimental results show the superior certified robustness of these Noised Diffusion Classifiers (NDCs).
Towards Transferable Targeted 3D Adversarial Attack in the Physical World
However, the field of transferable targeted 3D adversarial attacks remains vacant.
Focus on Hiders: Exploring Hidden Threats for Enhancing Adversarial Training
Adversarial training is often formulated as a min-max problem, however, concentrating only on the worst adversarial examples causes alternating repetitive confusion of the model, i. e., previously defended or correctly classified samples are not defensible or accurately classifiable in subsequent adversarial training.
Machine Vision Therapy: Multimodal Large Language Models Can Enhance Visual Robustness via Denoising In-Context Learning
Concretely, by estimating a transition matrix that captures the probability of one class being confused with another, an instruction containing a correct exemplar and an erroneous one from the most probable noisy class can be constructed.
Evil Geniuses: Delving into the Safety of LLM-based Agents
Rapid advancements in large language models (LLMs) have revitalized in LLM-based agents, exhibiting impressive human-like behaviors and cooperative capabilities in various scenarios.
How Robust is Google's Bard to Adversarial Image Attacks?
By attacking white-box surrogate vision encoders or MLLMs, the generated adversarial examples can mislead Bard to output wrong image descriptions with a 22% success rate based solely on the transferability.
Robustness and Generalizability of Deepfake Detection: A Study with Diffusion Models
The rise of deepfake images, especially of well-known personalities, poses a serious threat to the dissemination of authentic information.
Root Pose Decomposition Towards Generic Non-rigid 3D Reconstruction with Monocular Videos
The key idea of our method, Root Pose Decomposition (RPD), is to maintain a per-frame root pose transformation, meanwhile building a dense field with local transformations to rectify the root pose.
Improving Viewpoint Robustness for Visual Recognition via Adversarial Training
Experimental results show that VIAT significantly improves the viewpoint robustness of various image classifiers based on the diversity of adversarial viewpoints generated by GMVFool.
Towards Viewpoint-Invariant Visual Recognition via Adversarial Training
Visual recognition models are not invariant to viewpoint changes in the 3D world, as different viewing directions can dramatically affect the predictions given the same object.
Distributional Modeling for Location-Aware Adversarial Patches
In this paper, we propose the Distribution-Optimized Adversarial Patch (DOPatch), a novel method that optimizes a multimodal distribution of adversarial locations instead of individual ones.
Evaluating the Robustness of Text-to-image Diffusion Models against Real-world Attacks
Text-to-image (T2I) diffusion models (DMs) have shown promise in generating high-quality images from textual descriptions.
DIFFender: Diffusion-Based Adversarial Defense against Patch Attacks
Adversarial attacks, particularly patch attacks, pose significant threats to the robustness and reliability of deep learning models.
Robust Classification via a Single Diffusion Model
As RDC does not require training on particular adversarial attacks, we demonstrate that it is more generalizable to defend against multiple unseen threats.
Ranked #2 on
Adversarial Defense
on CIFAR-10
Text-to-Image Diffusion Models can be Easily Backdoored through Multimodal Data Poisoning
To gain a better understanding of the training process and potential risks of text-to-image synthesis, we perform a systematic investigation of backdoor attack on text-to-image diffusion models and propose BadT2I, a general multimodal backdoor attack framework that tampers with image synthesis in diverse semantic levels.
Understanding the Robustness of 3D Object Detection with Bird's-Eye-View Representations in Autonomous Driving
However, there still lacks a systematic understanding of the robustness of these vision-dependent BEV models, which is closely related to the safety of autonomous driving systems.
Towards Effective Adversarial Textured 3D Meshes on Physical Face Recognition
The goal of this work is to develop a more reliable technique that can carry out an end-to-end evaluation of adversarial robustness for commercial systems.
Compacting Binary Neural Networks by Sparse Kernel Selection
Binary Neural Network (BNN) represents convolution weights with 1-bit values, which enhances the efficiency of storage and computation.
Benchmarking Robustness of 3D Object Detection to Common Corruptions in Autonomous Driving
3D object detection is an important task in autonomous driving to perceive the surroundings.
Rethinking Model Ensemble in Transfer-based Adversarial Attacks
It is widely recognized that deep learning models lack robustness to adversarial examples.
GNOT: A General Neural Operator Transformer for Operator Learning
However, there are several challenges for learning operators in practical applications like the irregular mesh, multiple input functions, and complexity of the PDEs' solution.
A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking
In our benchmark, we evaluate the robustness of 55 typical deep learning models on ImageNet with diverse architectures (e. g., CNNs, Transformers) and learning algorithms (e. g., normal supervised training, pre-training, adversarial training) under numerous adversarial attacks and out-of-distribution (OOD) datasets.
Benchmarking Robustness of 3D Object Detection to Common Corruptions
3D object detection is an important task in autonomous driving to perceive the surroundings.
Artificial Intelligence Security Competition (AISC)
The security of artificial intelligence (AI) is an important research area towards safe, reliable, and trustworthy AI systems.
Improving transferability of 3D adversarial attacks with scale and shear transformations
Extensive experiments show that the SS attack proposed in this paper can be seamlessly combined with the existing state-of-the-art (SOTA) 3D point cloud attack methods to form more powerful attack methods, and the SS attack improves the transferability over 3. 6 times compare to the baseline.
Isometric 3D Adversarial Examples in the Physical World
For naturalness, we constrain the adversarial example to be $\epsilon$-isometric to the original one by adopting the Gaussian curvature as a surrogate metric guaranteed by a theoretical analysis.
ViewFool: Evaluating the Robustness of Visual Recognition to Adversarial Viewpoints
Recent studies have demonstrated that visual recognition models lack robustness to distribution shift.
Pre-trained Adversarial Perturbations
Self-supervised pre-training has drawn increasing attention in recent years due to its superior performance on numerous downstream tasks after fine-tuning.
GSmooth: Certified Robustness against Semantic Transformations via Generalized Randomized Smoothing
Under the GSmooth framework, we present a scalable algorithm that uses a surrogate image-to-image network to approximate the complex transformation.
Kallima: A Clean-label Framework for Textual Backdoor Attacks
Although Deep Neural Network (DNN) has led to unprecedented progress in various natural language processing (NLP) tasks, research shows that deep models are extremely vulnerable to backdoor attacks.
BadDet: Backdoor Attacks on Object Detection
We propose four kinds of backdoor attacks for object detection task: 1) Object Generation Attack: a trigger can falsely generate an object of the target class; 2) Regional Misclassification Attack: a trigger can change the prediction of a surrounding object to the target class; 3) Global Misclassification Attack: a single trigger can change the predictions of all objects in an image to the target class; and 4) Object Disappearance Attack: a trigger can make the detector fail to detect the object of the target class.
A Roadmap for Big Model
With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm.
Query-Efficient Black-box Adversarial Attacks Guided by a Transfer-based Prior
However, the existing methods inevitably suffer from low attack success rates or poor query efficiency since it is difficult to estimate the gradient in a high-dimensional input space with limited information.
Controllable Evaluation and Generation of Physical Adversarial Patch on Face Recognition
It is therefore imperative to develop a framework that can enable a comprehensive evaluation of the vulnerability of face recognition in the physical world.
Unrestricted Adversarial Attacks on ImageNet Competition
Many works have investigated the adversarial attacks or defenses under the settings where a bounded and imperceptible perturbation can be added to the input.
Adversarial Attacks on ML Defense Models Competition
Due to the vulnerability of deep neural networks (DNNs) to adversarial examples, a large number of defense techniques have been proposed to alleviate this problem in recent years.
Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness
The vulnerability of deep neural networks to adversarial examples has motivated an increasing number of defense strategies for promoting model robustness.
Boosting Transferability of Targeted Adversarial Examples via Hierarchical Generative Networks
Transfer-based adversarial attacks can evaluate model robustness in the black-box setting.
Improving Transferability of Adversarial Patches on Face Recognition with Generative Models
However, deep CNNs are vulnerable to adversarial patches, which are physically realizable and stealthy, raising new security concerns on the real-world applications of these models.
Accumulative Poisoning Attacks on Real-time Data
Collecting training data from untrusted sources exposes machine learning services to poisoning adversaries, who maliciously manipulate training data to degrade the model accuracy.
Exploring Memorization in Adversarial Training
In this paper, we explore the memorization effect in adversarial training (AT) for promoting a deeper understanding of model capacity, convergence, generalization, and especially robust overfitting of the adversarially trained models.
Two Coupled Rejection Metrics Can Tell Adversarial Examples Apart
Along with this routine, we find that confidence and a rectified confidence (R-Con) can form two coupled rejection metrics, which could provably distinguish wrongly classified inputs from correctly classified ones.
Automated Decision-based Adversarial Attacks
Deep learning models are vulnerable to adversarial examples, which can fool a target classifier by imposing imperceptible perturbations onto natural examples.
The art of defense: letting networks fool the attacker
To this end, in this paper, we propose a novel adversarial defense for 3D point cloud classifier that makes full use of the nature of the DNNs.
Black-box Detection of Backdoor Attacks with Limited Information and Data
Although deep neural networks (DNNs) have made rapid progress in recent years, they are vulnerable in adversarial environments.
Understanding and Exploring the Network with Stochastic Architectures
In this work, we decouple the training of a network with stochastic architectures (NSA) from NAS and provide a first systematical investigation on it as a stand-alone problem.
Bag of Tricks for Adversarial Training
Adversarial training (AT) is one of the most effective strategies for promoting model robustness.
BayesAdapter: Being Bayesian, Inexpensively and Robustly, via Bayesian Fine-tuning
Despite their theoretical appealingness, Bayesian neural networks (BNNs) are falling far behind in terms of adoption in real-world applications compared with normal NNs, mainly due to their limited scalability in training, and low fidelity in their uncertainty estimates.
RobFR: Benchmarking Adversarial Robustness on Face Recognition
Based on large-scale evaluations, the commercial FR API services fail to exhibit acceptable performance on robustness evaluation, and we also draw several important conclusions for understanding the adversarial robustness of FR models and providing insights for the design of robust FR models.
Benchmarking Adversarial Robustness on Image Classification
Deep neural networks are vulnerable to adversarial examples, which becomes one of the most important research problems in the development of deep learning.
Towards Face Encryption by Generating Adversarial Identity Masks
As billions of personal data being shared through social media and network, the data privacy and security have drawn an increasing attention.
Boosting Adversarial Training with Hypersphere Embedding
Adversarial training (AT) is one of the most effective defenses against adversarial attacks for deep learning models.
Adversarial Distributional Training for Robust Deep Learning
Adversarial training (AT) is among the most effective techniques to improve model robustness by augmenting training data with adversarial examples.
Benchmarking Adversarial Robustness
Deep neural networks are vulnerable to adversarial examples, which becomes one of the most important research problems in the development of deep learning.
Improving Black-box Adversarial Attacks with a Transfer-based Prior
We consider the black-box adversarial setting, where the adversary has to generate adversarial perturbations without access to the target models to compute gradients.
Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness
Previous work shows that adversarially robust generalization requires larger sample complexity, and the same dataset, e. g., CIFAR-10, which enables good standard accuracy may not suffice to train robust models.
Efficient Decision-based Black-box Adversarial Attacks on Face Recognition
In this paper, we evaluate the robustness of state-of-the-art face recognition models in the decision-based black-box attack setting, where the attackers have no access to the model parameters and gradients, but can only acquire hard-label predictions by sending queries to the target model.
Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks
In this paper, we propose a translation-invariant attack method to generate more transferable adversarial examples against the defense models.
Batch Virtual Adversarial Training for Graph Convolutional Networks
We present batch virtual adversarial training (BVAT), a novel regularization method for graph convolutional networks (GCNs).
Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples
3) We propose to improve the consistency of neurons on adversarial example subset by an adversarial training algorithm with a consistent loss.
Composite Binary Decomposition Networks
Binary neural networks have great resource and computing efficiency, while suffer from long training procedure and non-negligible accuracy drops, when comparing to the full-precision counterparts.
Learning Visual Knowledge Memory Networks for Visual Question Answering
Second is the mechanism for handling multiple knowledge facts expanding from question and answer pairs.
Adversarial Attacks and Defences Competition
To accelerate research on adversarial examples and robustness of machine learning classifiers, Google Brain organized a NIPS 2017 competition that encouraged researchers to develop new methods to generate adversarial examples as well as to develop new ways to defend against them.
Defense against Adversarial Attacks Using High-Level Representation Guided Denoiser
First, with HGD as a defense, the target model is more robust to either white-box or black-box adversarial attacks.
Boosting Adversarial Attacks with Momentum
To further improve the success rates for black-box attacks, we apply momentum iterative algorithms to an ensemble of models, and show that the adversarially trained models with a strong defense ability are also vulnerable to our black-box attacks.
Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples
We find that: (1) the neurons in DNNs do not truly detect semantic objects/parts, but respond to objects/parts only as recurrent discriminative patches; (2) deep visual representations are not robust distributed codes of visual concepts because the representations of adversarial images are largely not consistent with those of real images, although they have similar visual appearance, both of which are different from previous findings.
Learning Accurate Low-Bit Deep Neural Networks with Stochastic Quantization
This procedure can greatly compensate the quantization error and thus yield better accuracy for low-bit DNNs.
Towards Robust Detection of Adversarial Examples
Although the recent progress is substantial, deep learning methods can be vulnerable to the maliciously generated adversarial examples.
Improving Interpretability of Deep Neural Networks with Semantic Information
Interpretability of deep neural networks (DNNs) is essential since it enables users to understand the overall strengths and weaknesses of the models, conveys an understanding of how the models will behave in the future, and how to diagnose and correct potential problems.
Feature Engineering and Ensemble Modeling for Paper Acceptance Rank Prediction
Measuring research impact and ranking academic achievement are important and challenging problems.
Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
An adversarial example library for constructing attacks, building defenses, and benchmarking both
