Search Results for author:
Found 98 papers, 41 papers with code
Does Machine Unlearning Truly Remove Model Knowledge? A Framework for Auditing Unlearning in LLMs
In this work, we introduce a comprehensive auditing framework for unlearning evaluation, comprising three benchmark datasets, six unlearning algorithms, and five prompt-based auditing methods.
DD-Ranking: Rethinking the Evaluation of Dataset Distillation
In recent years, dataset distillation has provided a reliable solution for data compression, where models trained on the resulting smaller synthetic datasets achieve performance comparable to those trained on the original datasets.
A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment
Currently, existing surveys on LLM safety primarily focus on specific stages of the LLM lifecycle, e. g., deployment phase or fine-tuning phase, lacking a comprehensive understanding of the entire "lifechain" of LLMs.
REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websites
We introduce REAL, a benchmark and framework for multi-turn agent evaluations on deterministic simulations of real-world websites.
Review, Refine, Repeat: Understanding Iterative Decoding of AI Agents with Dynamic Evaluation and Selection
Hence, we propose Iterative Agent Decoding (IAD) which combines iterative refinement with dynamic candidate evaluation and selection guided by a verifier.
ShieldGemma 2: Robust and Tractable Image Content Moderation
We introduce ShieldGemma 2, a 4B parameter image content moderation model built on Gemma 3.
Exploring Typographic Visual Prompts Injection Threats in Cross-Modality Generation Models
To better observe performance modifications and characteristics of this threat, we also introduce the TVPI Dataset.
Magnet: Multi-turn Tool-use Data Synthesis and Distillation via Graph Translation
To address this, we propose Magnet, a principled framework for synthesizing high-quality training trajectories to enhance the function calling capability of large language model agents in multi-turn conversations with humans.
Improving Adversarial Transferability in MLLMs via Dynamic Vision-Language Alignment Attack
Multimodal Large Language Models (MLLMs), built upon LLMs, have recently gained attention for their capabilities in image recognition and understanding.
PlanGEN: A Multi-Agent Framework for Generating Planning and Reasoning Trajectories for Complex Problem Solving
Recent agent frameworks and inference-time algorithms often struggle with complex planning problems due to limitations in verifying generated plans or reasoning and varying complexity of instances within a single task.
Safety at Scale: A Comprehensive Survey of Large Model Safety
The rapid advancement of large models, driven by their exceptional abilities in learning and generalization through large-scale pre-training, has reshaped the landscape of Artificial Intelligence (AI).
Jailbreak-AudioBench: In-Depth Evaluation and Analysis of Jailbreak Threats for Large Audio Language Models
Integrating various modality encoders further expands their capabilities, giving rise to Multimodal Large Language Models (MLLMs) that process not only text but also visual and auditory modality inputs.
FocalPO: Enhancing Preference Optimizing by Focusing on Correct Preference Rankings
These algorithms implicitly treat the LLM as a reward model, and focus on training it to correct misranked preference pairs.
Open Eyes, Then Reason: Fine-grained Visual Mathematical Understanding in MLLMs
Current multimodal large language models (MLLMs) often underperform on mathematical problem-solving tasks that require fine-grained visual understanding.
SafetyDPO: Scalable Safety Alignment for Text-to-Image Generation
Using a custom DPO strategy and this dataset, we train safety experts, in the form of low-rank adaptation (LoRA) matrices, able to guide the generation process away from specific safety-related concepts.
Not Just Text: Uncovering Vision Modality Typographic Threats in Image Generation Models
Current image generation models can effortlessly produce high-quality, highly realistic images, but this also increases the risk of misuse.
Benchmarking Open-ended Audio Dialogue Understanding for Large Audio-Language Models
Recent advances, such as GPT-4o, have enabled LALMs in back-and-forth audio dialogues with humans.
UVCG: Leveraging Temporal Consistency for Universal Video Protection
The security risks of AI-driven video editing have garnered significant attention.
ReVisionLLM: Recursive Vision-Language Model for Temporal Grounding in Hour-Long Videos
We propose ReVisionLLM, a recursive vision-language model designed to locate events in hour-long videos.
FedBiP: Heterogeneous One-Shot Federated Learning with Personalized Latent Diffusion Models
However, directly applying pretrained LDM to heterogeneous OSFL results in significant distribution shifts in synthetic data, leading to performance degradation in classification models trained on such data.
Visual Question Decomposition on Multimodal Large Language Models
The finetuning pipeline consists of our proposed dataset and a training objective for selective decomposition.
Multimodal Pragmatic Jailbreak on Text-to-image Models
This work introduces a novel type of jailbreak, which triggers T2I models to generate the image with visual text, where the image and the text, although considered to be safe in isolation, combine to form unsafe content.
Manipulation Facing Threats: Evaluating Physical Vulnerabilities in End-to-End Vision Language Action Models
Recently, driven by advancements in Multimodal Large Language Models (MLLMs), Vision Language Action Models (VLAMs) are being proposed to achieve better performance in open-vocabulary scenarios for robotic manipulation tasks.
HTS-Attack: Heuristic Token Search for Jailbreaking Text-to-Image Models
Text-to-Image(T2I) models have achieved remarkable success in image generation and editing, yet these models still have many potential issues, particularly in generating inappropriate or Not-Safe-For-Work(NSFW) content.
Can Editing LLMs Inject Harm?
Then, we find that editing attacks can inject both types of misinformation into LLMs, and the effectiveness is particularly high for commonsense misinformation injection.
Dataset Distillation by Automatic Training Trajectories
Dataset Distillation is used to create a concise, yet informative, synthetic dataset that can replace the original dataset for training purposes.
MMRo: Are Multimodal LLMs Eligible as the Brain for In-Home Robotics?
It is fundamentally challenging for robots to serve as useful assistants in human environments because this requires addressing a spectrum of sub-problems across robotics, including perception, language understanding, reasoning, and planning.
Localizing Events in Videos with Multimodal Queries
Localizing events in videos based on semantic queries is a pivotal task in video understanding, with the growing significance of user-oriented applications like video search.
Provably Better Explanations with Optimized Aggregation of Feature Attributions
Using feature attributions for post-hoc explanations is a common practice to understand and verify the predictions of opaque machine learning models.
Learning Visual Prompts for Guiding the Attention of Vision Transformers
The learned visual prompt, added to any input image would redirect the attention of the pre-trained vision transformer to its spatial location on the image.
Improved Techniques for Optimization-Based Jailbreaking on Large Language Models
Many red-teaming efforts aim to jailbreak LLMs, where among these efforts, the Greedy Coordinate Gradient (GCG) attack's success has led to a growing interest in the study of optimization-based jailbreaking techniques.
Typography Leads Semantic Diversifying: Amplifying Adversarial Transferability across Multimodal Large Language Models
Recently, Multimodal Large Language Models (MLLMs) achieve remarkable performance in numerous zero-shot tasks due to their outstanding cross-modal interaction and comprehension abilities.
When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models
Hence, with this paper, we aim to chart a course for future research that explores and expands the capabilities of 3D-LLMs in understanding and interacting with the complex 3D world.
Special Characters Attack: Toward Scalable Training Data Extraction From Large Language Models
In this paper, we take a step further and show that certain special characters or their combinations with English letters are stronger memory triggers, leading to more severe data leakage.
Energy-Latency Manipulation of Multi-modal Large Language Models via Verbose Samples
For verbose videos, a frame feature diversity loss is proposed to increase the feature diversity among frames.
Latent Guard: a Safety Framework for Text-to-image Generation
Hence, we propose Latent Guard, a framework designed to improve safety measures in text-to-image generation.
A Survey on Responsible Generative AI: What to Generate and What Not
To answer the question, this paper investigates the practical responsible requirements of both textual and visual generative models, outlining five key considerations: generating truthful content, avoiding toxic content, refusing harmful instruction, leaking no training data-related content, and ensuring generated content identifiable.
Red Teaming GPT-4V: Are GPT-4V Safe Against Uni/Multi-Modal Jailbreak Attacks?
Various jailbreak attacks have been proposed to red-team Large Language Models (LLMs) and revealed the vulnerable safeguards of LLMs.
Which Model Generated This Image? A Model-Agnostic Approach for Origin Attribution
In this work, we study the origin attribution of generated images in a practical setting where only a few images generated by a source model are available and the source model cannot be accessed.
As Firm As Their Foundations: Can open-sourced foundation models be used to create adversarial examples for downstream tasks?
Foundation models pre-trained on web-scale vision-language data, such as CLIP, are widely used as cornerstones of powerful machine learning systems.
An Image Is Worth 1000 Lies: Adversarial Transferability across Prompts on Vision-Language Models
Given that VLMs rely on prompts to adapt to different tasks, an intriguing question emerges: Can a single adversarial image mislead all predictions of VLMs when a thousand different prompts are given?
Hide in Thicket: Generating Imperceptible and Rational Adversarial Perturbations on 3D Point Clouds
We find that concealing deformation perturbations in areas insensitive to human eyes can achieve a better trade-off between imperceptibility and adversarial strength, specifically in parts of the object surface that are complex and exhibit drastic curvature changes.
Unveiling Typographic Deceptions: Insights of the Typographic Vulnerability in Large Vision-Language Model
Large Vision-Language Models (LVLMs) rely on vision encoders and Large Language Models (LLMs) to exhibit remarkable capabilities on various multi-modal tasks in the joint space of vision and language.
Stop Reasoning! When Multimodal LLM with Chain-of-Thought Reasoning Meets Adversarial Image
Based on our findings, we further propose a novel attack method, termed as stop-reasoning attack, that attacks the model while bypassing the CoT reasoning process.
Can Large Language Model Agents Simulate Human Trust Behavior?
In this paper, we focus on one critical and elemental behavior in human interactions, trust, and investigate whether LLM agents can simulate human trust behavior.
Inducing High Energy-Latency of Large Vision-Language Models with Verbose Images
Once attackers maliciously induce high energy consumption and latency time (energy-latency cost) during inference of VLMs, it will exhaust computational resources.
Does Few-shot Learning Suffer from Backdoor Attacks?
However, in this paper, we propose the Few-shot Learning Backdoor Attack (FLBA) to show that FSL can still be vulnerable to backdoor attacks.
XAI for In-hospital Mortality Prediction via Multimodal ICU Data
To address this issue, this paper proposes an eXplainable Multimodal Mortality Predictor (X-MMP) approaching an efficient, explainable AI solution for predicting in-hospital mortality via multimodal ICU data.
Initialization Matters for Adversarial Transfer Learning
Based on this, we propose Robust Linear Initialization (RoLI) for adversarial finetuning, which initializes the linear head with the weights obtained by adversarial linear probing to maximally inherit the robustness from pretraining.
OT-Attack: Enhancing Adversarial Transferability of Vision-Language Models via Optimal Transport Optimization
Investigating the generation of high-transferability adversarial examples is crucial for uncovering VLP models' vulnerabilities in practical scenarios.
TranSegPGD: Improving Transferability of Adversarial Examples on Semantic Segmentation
At the second stage, the pixels are divided into different branches based on their transferable property which is dependent on Kullback-Leibler divergence.
Improving Adversarial Transferability via Model Alignment
During the alignment process, the parameters of the source model are fine-tuned to minimize an alignment loss.
Can Multimodal Large Language Models Truly Perform Multimodal In-Context Learning?
Experiments revealed that multimodal ICL is predominantly driven by the textual content whereas the visual information in the demos has little influence.
MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models
The security concerns surrounding Large Language Models (LLMs) have been extensively explored, yet the safety of Multimodal Large Language Models (MLLMs) remains understudied.
Self-Discovering Interpretable Diffusion Latent Directions for Responsible Text-to-Image Generation
A risk with these models is the potential generation of inappropriate content, such as biased or harmful images.
Benchmarking Robustness of Text-Image Composed Retrieval
In this paper, we perform the first robustness study and establish three new diversified benchmarks for systematic analysis of text-image composed retrieval against natural corruptions in both vision and text and further probe textural understanding.
SPOT! Revisiting Video-Language Models for Event Understanding
This raises a question: with such weak supervision, can video representation in video-language models gain the ability to distinguish even factual discrepancies in textual description and understand fine-grained events?
A Survey on Transferability of Adversarial Examples across Deep Neural Networks
This survey explores the landscape of the adversarial transferability of adversarial examples.
Fast Propagation is Better: Accelerating Single-Step Adversarial Training via Sampling Subnetworks
Besides, we provide theoretical analysis to show the model robustness can be improved by the single-step adversarial training with sampled subnetworks.
Boosting Fair Classifier Generalization through Adaptive Priority Reweighing
Our adaptive reweighing method prioritizes samples closer to the decision boundary and assigns a higher weight to improve the generalizability of fair classifiers.
Exploring Non-additive Randomness on ViT against Query-Based Black-Box Attacks
In this work, we first taxonomize the stochastic defense strategies against QBBA.
Revisiting and Exploring Efficient Fast Adversarial Training via LAW: Lipschitz Regularization and Auto Weight Averaging
In this paper, we conduct a comprehensive study of over 10 fast adversarial training methods in terms of adversarial robustness and training costs.
Multi-event Video-Text Retrieval
In this study, we introduce the Multi-event Video-Text Retrieval (MeVTR) task, addressing scenarios in which each video contains multiple different events, as a niche scenario of the conventional Video-Text Retrieval Task.
FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning
FedDAT is the first approach that enables an efficient distributed finetuning of foundation models for a variety of heterogeneous Vision-Language tasks.
Discretization-Induced Dirichlet Posterior for Robust Uncertainty Quantification on Regression
In this work, we propose a generalized AuxUE scheme for more robust uncertainty quantification on regression tasks.
FedPop: Federated Population-based Hyperparameter Tuning
This is mainly because their "training-after-tuning" framework is unsuitable for FL with limited client computation power.
A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models
This paper aims to provide a comprehensive survey of cutting-edge research in prompt engineering on three types of vision-language models: multimodal-to-text generation models (e. g. Flamingo), image-text matching models (e. g.
Reliable Evaluation of Adversarial Transferability
Adversarial examples (AEs) with small adversarial perturbations can mislead deep neural networks (DNNs) into wrong predictions.
Towards Robust Prompts on Vision-Language Models
With the advent of vision-language models (VLMs) that can perform in-context and prompt-based learning, how can we design prompting approaches that robustly generalize to distribution shift and can be used on novel classes outside the support set of the prompts?
Backdoor Defense via Adaptively Splitting Poisoned Dataset
With the split clean data pool and polluted data pool, ASD successfully defends against backdoor attacks during training.
Influencer Backdoor Attack on Semantic Segmentation
In this work, we explore backdoor attacks on segmentation models to misclassify all pixels of a victim class by injecting a specific trigger on non-victim pixels during inferences, which is dubbed Influencer Backdoor Attack (IBA).
Explainability and Robustness of Deep Visual Classification Models
The vulnerability of deep neural networks poses challenges to current visual classification models.
Do DALL-E and Flamingo Understand Each Other?
To study this question, we propose a reconstruction task where Flamingo generates a description for a given image and DALL-E uses this description as input to synthesize a new image.
CL-CrossVQA: A Continual Learning Benchmark for Cross-Domain Visual Question Answering
Visual Question Answering (VQA) is a multi-discipline research task.
SegPGD: An Effective and Efficient Adversarial Attack for Evaluating and Boosting Segmentation Robustness
Since SegPGD can create more effective adversarial examples, the adversarial training with our SegPGD can boost the robustness of segmentation models.
Towards Efficient Adversarial Training on Vision Transformers
Vision Transformer (ViT), as a powerful alternative to Convolutional Neural Network (CNN), has received much attention.
Watermark Vaccine: Adversarial Attacks to Prevent Watermark Removal
Inspired by the vulnerability of DNNs on adversarial perturbations, we propose a novel defence mechanism by adversarial machine learning for good.
FRAug: Tackling Federated Learning with Non-IID Features via Representation Augmentation
Real-world applications usually involve a distribution shift across the datasets of the different clients, which hurts the generalization ability of the clients to unseen samples from their respective data distributions.
ECOLA: Enhanced Temporal Knowledge Embeddings with Contextualized Language Representations
Since conventional knowledge embedding models cannot take full advantage of the abundant textual information, there have been extensive research efforts in enhancing knowledge embedding using texts.
Adversarial Examples on Segmentation Models Can be Easy to Transfer
The high transferability achieved by our method shows that, in contrast to the observations in previous work, adversarial examples on a segmentation model can be easy to transfer to other segmentation models.
Are Vision Transformers Robust to Patch Perturbations?
However, when ViTs are attacked by an adversary, the attention mechanism can be easily fooled to focus more on the adversarially perturbed patches and cause a mistake.
Are Vision Transformers Robust to Patch-wise Perturbations?
Based on extensive qualitative and quantitative experiments, we discover that ViT's stronger robustness to natural corrupted patches and higher vulnerability against adversarial patches are both caused by the attention mechanism.
Simple Distillation Baselines for Improving Small Self-supervised Models
While large self-supervised models have rivalled the performance of their supervised counterparts, small models still struggle.
Attacking Adversarial Attacks as A Defense
In this work, we find that the adversarial attacks can also be vulnerable to small perturbations.
Quantifying Predictive Uncertainty in Medical Image Analysis with Deep Kernel Learning
We propose an uncertainty-aware deep kernel learning model which permits the estimation of the uncertainty in the prediction by a pipeline of a Convolutional Neural Network and a sparse Gaussian Process.
Capsule Network is Not More Robust than Convolutional Network
The examination reveals five major new/different components in CapsNet: a transformation process, a dynamic routing layer, a squashing function, a marginal loss other than cross-entropy loss, and an additional class-conditional reconstruction loss for regularization.
Effective and Efficient Vote Attack on Capsule Networks
As alternatives to CNNs, the recently proposed Capsule Networks (CapsNets) are shown to be more robust to white-box attacks than CNNs under popular attack protocols.
Interpretable Graph Capsule Networks for Object Recognition
In the proposed model, individual classification explanations can be created effectively and efficiently.
Introspective Learning by Distilling Knowledge from Online Self-explanation
Motivated by the conclusion, we propose an implementation of introspective learning by distilling knowledge from online self-explanations.
Search for Better Students to Learn Distilled Knowledge
The knowledge of a well-performed teacher is distilled to a student with a small architecture.
Neural Network Memorization Dissection
What is the difference between DNNs trained with random labels and the ones trained with true labels?
Improving the Robustness of Capsule Networks to Image Affine Transformations
Our investigation reveals that the routing procedure contributes neither to the generalization ability nor to the affine robustness of the CapsNets.
Semantics for Global and Local Interpretation of Deep Neural Networks
Deep neural networks (DNNs) with high expressiveness have achieved state-of-the-art performance in many tasks.
Contextual Prediction Difference Analysis for Explaining Individual Image Classifications
In this work, we first show that PDA can suffer from saturated classifiers.
Understanding Bias in Machine Learning
Bias is known to be an impediment to fair decisions in many domains such as human resources, the public sector, health care etc.
Saliency Methods for Explaining Adversarial Attacks
The idea behind saliency methods is to explain the classification decisions of neural networks by creating so-called saliency maps.
Understanding Individual Decisions of CNNs via Contrastive Backpropagation
The experiments and analysis conclude that the explanations generated by LRP are not class-discriminative.
Semi-supervised Outlier Detection using Generative And Adversary Framework
In the adversarial process of training CorGAN, the Generator is supposed to generate outlier samples for negative class, and the Discriminator as an one-class classifier is trained to distinguish data from training datasets (i. e. positive class) and generated data from the Generator (i. e. negative class).
