Search Results for author:
Found 15 papers, 9 papers with code
EVA: Red-Teaming GUI Agents via Evolving Indirect Prompt Injection
As multimodal agents are increasingly trained to operate graphical user interfaces (GUIs) to complete user tasks, they face a growing threat from indirect prompt injection, attacks in which misleading instructions are embedded into the agent's visual environment, such as popups or chat messages, and misinterpreted as part of the intended task.
Hidden Ghost Hand: Unveiling Backdoor Vulnerabilities in MLLM-Powered Mobile GUI Agents
Graphical user interface (GUI) agents powered by multimodal large language models (MLLMs) have shown greater promise for human-interaction.
On Path to Multimodal Generalist: General-Level and General-Bench
This project introduces General-Level, an evaluation framework that defines 5-scale levels of MLLM performance and generality, offering a methodology to compare MLLMs and gauge the progress of existing systems towards more robust multimodal generalists and, ultimately, towards AGI.
Probing then Editing Response Personality of Large Language Models
We conduct probing experiments on 11 open-source LLMs over the PersonalityEdit benchmark and find that LLMs predominantly encode personality for responding in their middle and upper layers, with instruction-tuned models demonstrating a slightly clearer separation of personality traits.
Watch Out Your Album! On the Inadvertent Privacy Memorization in Multi-Modal Large Language Models
Multi-Modal Large Language Models (MLLMs) have exhibited remarkable performance on various vision-language tasks such as Visual Question Answering (VQA).
Smoothing Grounding and Reasoning for MLLM-Powered GUI Agents with Query-Oriented Pivot Tasks
Perception-enhanced pre-training, particularly through grounding techniques, is widely adopted to enhance the performance of graphical user interface (GUI) agents.
Investigating the Adaptive Robustness with Knowledge Conflicts in LLM-based Multi-Agent Systems
Recent advances in Large Language Models (LLMs) have upgraded them from sophisticated text generators to autonomous agents capable of corporation and tool use in multi-agent systems (MASs).
NSmark: Null Space Based Black-box Watermarking Defense Framework for Language Models
Language models (LMs) have emerged as critical intellectual property (IP) assets that necessitate protection.
Flooding Spread of Manipulated Knowledge in LLM-Based Multi-Agent Communities
The rapid adoption of large language models (LLMs) in multi-agent systems has highlighted their impressive capabilities in various applications, such as collaborative problem-solving and autonomous negotiation.
TrojanRAG: Retrieval-Augmented Generation Can Be Backdoor Driver in Large Language Models
To improve the recall of the RAG for the target contexts, we introduce a knowledge graph to construct structured data to achieve hard matching at a fine-grained level.
General LLMs as Instructors for Domain-Specific LLMs: A Sequential Fusion Method to Integrate Extraction and Editing
To tackle this challenge, we introduce a Sequential Fusion method to integrate knowledge from complex contexts into LLMs.
Federated Semi-supervised Learning for Medical Image Segmentation with intra-client and inter-client Consistency
The intra-client and inter-client consistency learning are introduced to smooth predictions at the data level and avoid confirmation bias of local models.
How Large Language Models Encode Context Knowledge? A Layer-Wise Probing Study
Previous work has showcased the intriguing capability of large language models (LLMs) in retrieving facts and processing context knowledge.
Investigating Multi-Hop Factual Shortcuts in Knowledge Editing of Large Language Models
Recent work has showcased the powerful capability of large language models (LLMs) in recalling knowledge and reasoning.
On the Robustness of Editing Large Language Models
RQ1: Can edited LLMs behave consistently resembling communicative AI in realistic situations?
