Search Results for author:
Found 18 papers, 11 papers with code
OET: Optimization-based prompt injection Evaluation Toolkit
Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language understanding and generation, enabling their widespread adoption across various domains.
Doxing via the Lens: Revealing Privacy Leakage in Image Geolocation for Agentic Multi-Modal Large Reasoning Model
The increasing capabilities of agentic multi-modal large reasoning models, such as ChatGPT o3, have raised critical concerns regarding privacy leakage through inadvertent image geolocation.
AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detection
The rapid advancements in Large Language Models (LLMs) have enabled their deployment as autonomous agents for handling complex tasks in dynamic environments.
InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Models
NotInject contains 339 benign samples enriched with trigger words common in prompt injection attacks, enabling fine-grained evaluation.
RePD: Defending Jailbreak Attack through a Retrieval-based Prompt Decomposition Process
In this study, we introduce RePD, an innovative attack Retrieval-based Prompt Decomposition framework designed to mitigate the risk of jailbreak attacks on large language models (LLMs).
AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs
In this paper, we propose AutoDAN-Turbo, a black-box jailbreak method that can automatically discover as many jailbreak strategies as possible from scratch, without any human intervention or predefined scopes (e. g., specified candidate strategies), and use them for red-teaming.
MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding
We introduce MuirBench, a comprehensive benchmark that focuses on robust multi-image understanding capabilities of multimodal LLMs.
Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Character
With the advent and widespread deployment of Multimodal Large Language Models (MLLMs), ensuring their safety has become increasingly critical.
JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks
With the rapid advancements in Multimodal Large Language Models (MLLMs), securing these models against malicious inputs while aligning them with human values has emerged as a critical challenge.
Don't Listen To Me: Understanding and Exploring Jailbreak Prompts of Large Language Models
Building on the insights from the user study, we also developed a system using AI as the assistant to automate the process of jailbreak prompt generation.
AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting
However, with the integration of additional modalities, MLLMs are exposed to new vulnerabilities, rendering them prone to structured-based jailbreak attacks, where semantic content (e. g., "harmful text") has been injected into the images to mislead MLLMs.
Automatic and Universal Prompt Injection Attacks against Large Language Models
Large Language Models (LLMs) excel in processing and generating human language, powered by their ability to interpret and follow instructions.
DeceptPrompt: Exploiting LLM-driven Code Generation via Adversarial Natural Language Instructions
In this paper, we introduce DeceptPrompt, a novel algorithm that can generate adversarial natural language instructions that drive the Code LLMs to generate functionality correct code with vulnerabilities.
AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
In light of these challenges, we intend to answer this question: Can we develop an approach that can automatically generate stealthy jailbreak prompts?
Why Does Little Robustness Help? Understanding and Improving Adversarial Transferability from Surrogate Training
Building on these insights, we explore the impacts of data augmentation and gradient regularization on transferability and identify that the trade-off generally exists in the various training mechanisms, thus building a comprehensive blueprint for the regulation mechanism behind transferability.
Detecting Backdoors During the Inference Stage Based on Corruption Robustness Consistency
Deep neural networks are proven to be vulnerable to backdoor attacks.
Towards Efficient Data-Centric Robust Machine Learning with Noise-based Augmentation
The data-centric machine learning aims to find effective ways to build appropriate datasets which can improve the performance of AI models.
Protecting Facial Privacy: Generating Adversarial Identity Masks via Style-robust Makeup Transfer
While deep face recognition (FR) systems have shown amazing performance in identification and verification, they also arouse privacy concerns for their excessive surveillance on users, especially for public face images widely spread on social networks.
