Search Results for author:
Found 35 papers, 12 papers with code
CDR-Agent: Intelligent Selection and Execution of Clinical Decision Rules Using Large Language Model Agents
Clinical decision-making is inherently complex and fast-paced, particularly in emergency departments (EDs) where critical, rapid and high-stakes decisions are made.
SOSBENCH: Benchmarking Safety Alignment on Scientific Knowledge
Large language models (LLMs) exhibit advancing capabilities in complex tasks, such as reasoning and graduate-level question answering, yet their resilience against misuse, particularly involving scientifically sophisticated risks, remains underexplored.
How Memory Management Impacts LLM Agents: An Empirical Study of Experience-Following Behavior
In this paper, we conduct an empirical study on how memory management choices impact the LLM agents' behavior, especially their long-term performance.
Doxing via the Lens: Revealing Location-related Privacy Leakage on Multi-modal Large Reasoning Models
In this paper, we identify a novel category of privacy leakage in MLRMs: Adversaries can infer sensitive geolocation information, such as a user's home address or neighborhood, from user-generated images, including selfies captured in private settings.
Large Language Model Empowered Privacy-Protected Framework for PHI Annotation in Clinical Notes
By fine-tuning LLMs locally with synthetic notes, LPPA ensures strong privacy protection and high PHI annotation accuracy.
MMDT: Decoding the Trustworthiness and Safety of Multimodal Foundation Models
In this paper, we present the first unified platform, MMDT (Multimodal DecodingTrust), designed to provide a comprehensive safety and trustworthiness evaluation for MMFMs.
A Practical Memory Injection Attack against LLM Agents
During the injection of the malicious record, we propose an indication prompt to guide the agent to autonomously generate our designed bridging steps.
Multi-Faceted Studies on Data Poisoning can Advance LLM Development
While prior research on data poisoning attacks has primarily focused on the safety vulnerabilities of LLMs, these attacks face significant challenges in practice.
SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilities
Current research on large language model (LLM) safety usually focuses on short-answer responses, overlooking the long CoT style outputs of LRMs.
Unveiling Privacy Risks in LLM Agent Memory
Large Language Model (LLM) agents have become increasingly prevalent across various real-world applications.
SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents
With the integration of large language models (LLMs), embodied agents have strong capabilities to understand and plan complicated natural language instructions.
Data Free Backdoor Attacks
Existing backdoor attacks require either retraining the classifier with some clean data or modifying the model's architecture.
Are We There Yet? Revealing the Risks of Utilizing Large Language Models in Scholarly Peer Review
However, the unchecked adoption of LLMs poses significant risks to the integrity of the peer review system.
Towards Next-Generation Medical Agent: How o1 is Reshaping Decision-Making in Medical Scenarios
Artificial Intelligence (AI) has become essential in modern healthcare, with large language models (LLMs) offering promising advances in clinical decision-making.
Evaluation of OpenAI o1: Opportunities and Challenges of AGI
-Impressive performance in chip design tasks, outperforming specialized models in areas such as EDA script generation and bug analysis.
AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases
In particular, we form the trigger generation process as a constrained optimization to optimize backdoor triggers by mapping the triggered instances to a unique embedding space, so as to ensure that whenever a user instruction contains the optimized backdoor trigger, the malicious demonstrations are retrieved from the poisoned memory or knowledge base with high probability.
GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning
We also show that GuardAgent is able to define novel functions in adaption to emergent LLM agents and guard requests, which underscores its strong generalization capabilities.
ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs
In this paper, we propose a novel ASCII art-based jailbreak attack and introduce a comprehensive benchmark Vision-in-Text Challenge (ViTC) to evaluate the capabilities of LLMs in recognizing prompts that cannot be solely interpreted by semantics.
BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models
Moreover, we show that LLMs endowed with stronger reasoning capabilities exhibit higher susceptibility to BadChain, exemplified by a high average attack success rate of 97. 0% across the six benchmark tasks on GPT-4.
CBD: A Certified Backdoor Detector Based on Local Dominant Probability
Notably, for backdoor attacks with random perturbation triggers bounded by $\ell_2\leq0. 75$ which achieves more than 90\% attack success rate, CBD achieves 100\% (98\%), 100\% (84\%), 98\% (98\%), and 72\% (40\%) empirical (certified) detection true positive rates on the four benchmark datasets GTSRB, SVHN, CIFAR-10, and TinyImageNet, respectively, with low false positive rates.
Backdoor Mitigation by Correcting the Distribution of Neural Activations
Backdoor (Trojan) attacks are an important type of adversarial exploit against deep neural networks (DNNs), wherein a test instance is (mis)classified to the attacker's target class whenever the attacker's backdoor trigger is present.
Improved Activation Clipping for Universal Backdoor Mitigation and Test-Time Detection
Deep neural networks are vulnerable to backdoor attacks (Trojans), where an attacker poisons the training set with backdoor triggers so that the neural network learns to classify test-time triggers to the attacker's designated target class.
UMD: Unsupervised Model Detection for X2X Backdoor Attacks
Backdoor (Trojan) attack is a common threat to deep neural networks, where samples from one or more source classes embedded with a backdoor trigger will be misclassified to adversarial target classes.
MM-BD: Post-Training Detection of Backdoor Attacks with Arbitrary Backdoor Pattern Types Using a Maximum Margin Statistic
Our detector leverages the influence of the backdoor attack, independent of the backdoor embedding mechanism, on the landscape of the classifier's outputs prior to the softmax layer.
Post-Training Detection of Backdoor Attacks for Two-Class and Multi-Attack Scenarios
We show that our ET statistic is effective {\it using the same detection threshold}, irrespective of the classification domain, the attack configuration, and the BP reverse-engineering algorithm that is used.
Test-Time Detection of Backdoor Triggers for Poisoned Deep Neural Networks
A DNN being attacked will predict to an attacker-desired target class whenever a test sample from any source class is embedded with a backdoor pattern; while correctly classifying clean (attack-free) test samples.
Detecting Backdoor Attacks Against Point Cloud Classifiers
Backdoor attacks (BA) are an emerging threat to deep neural network classifiers.
A BIC-based Mixture Model Defense against Data Poisoning Attacks on Classifiers
Data Poisoning (DP) is an effective attack that causes trained classifiers to misclassify their inputs.
L-RED: Efficient Post-Training Detection of Imperceptible Backdoor Attacks without Access to the Training Set
Unfortunately, most existing REDs rely on an unrealistic assumption that all classes except the target class are source classes of the attack.
Reverse Engineering Imperceptible Backdoor Attacks on Deep Neural Networks for Detection and Training Set Cleansing
The attacker poisons the training set with a relatively small set of images from one (or several) source class(es), embedded with a backdoor pattern and labeled to a target class.
Revealing Perceptible Backdoors, without the Training Set, via the Maximum Achievable Misclassification Fraction Statistic
Here, we address post-training detection of innocuous perceptible backdoors in DNN image classifiers, wherein the defender does not have access to the poisoned training set, but only to the trained classifier, as well as unpoisoned examples.
Notes on Margin Training and Margin p-Values for Deep Neural Network Classifiers
We provide a new local class-purity theorem for Lipschitz continuous DNN classifiers.
Detection of Backdoors in Trained Classifiers Without Access to the Training Set
Here we address post-training detection of backdoor attacks in DNN image classifiers, seldom considered in existing works, wherein the defender does not have access to the poisoned training set, but only to the trained classifier itself, as well as to clean examples from the classification domain.
Adversarial Learning in Statistical Classification: A Comprehensive Review of Defenses Against Attacks
After introducing relevant terminology and the goals and range of possible knowledge of both attackers and defenders, we survey recent work on test-time evasion (TTE), data poisoning (DP), and reverse engineering (RE) attacks and particularly defenses against same.
A Mixture Model Based Defense for Data Poisoning Attacks Against Naive Bayes Spam Filters
Such attacks are successful mainly because of the poor representation power of the naive Bayes (NB) model, with only a single (component) density to represent spam (plus a possible attack).
