Search Results for author:
Found 10 papers, 5 papers with code
Iris: Breaking GUI Complexity with Adaptive Focus and Self-Refining
Digital agents are increasingly employed to automate tasks in interactive digital environments such as web pages, software applications, and operating systems.
STEP: Enhancing Video-LLMs' Compositional Reasoning by Spatio-Temporal Graph-guided Self-Training
Video Large Language Models (Video-LLMs) have recently shown strong performance in basic video understanding tasks, such as captioning and coarse-grained question answering, but struggle with compositional reasoning that requires multi-step spatio-temporal inference across object relations, interactions, and events.
AnyEdit: Mastering Unified High-Quality Image Editing for Any Idea
Instruction-based image editing aims to modify specific image elements with natural language instructions.
Unified Generative and Discriminative Training for Multi-modal Large Language Models
Discriminative training, exemplified by models like CLIP, excels in zero-shot image-text classification and retrieval, yet struggles with complex scenarios requiring fine-grained semantic differentiation.
Towards Unified Multimodal Editing with Enhanced Knowledge Collaboration
In this paper, we propose UniKE, a novel multimodal editing method that establishes a unified perspective and paradigm for intrinsic knowledge editing and external knowledge resorting.
Auto-Encoding Morph-Tokens for Multimodal LLM
For multimodal LLMs, the synergy of visual comprehension (textual output) and generation (visual output) presents an ongoing challenge.
Improving Vision Anomaly Detection with the Guidance of Language Modality
To learn a more compact latent space for the vision anomaly detector, CMLE learns a correlation structure matrix from the language modality, and then the latent space of vision modality will be learned with the guidance of the matrix.
I3: Intent-Introspective Retrieval Conditioned on Instructions
Recent studies indicate that dense retrieval models struggle to perform well on a wide variety of retrieval tasks that lack dedicated training data, as different retrieval tasks often entail distinct search intents.
Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructions
This shortcoming results in MLLMs' underperformance in comprehending demonstrative instructions consisting of multiple, interleaved, and multimodal instructions that demonstrate the required context to complete a task.
Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalization
Though effective, prompt tuning under few-shot settings on the one hand heavily relies on a good initialization of soft prompts.
