Search Results for author:
Found 10 papers, 9 papers with code
GROUNDHOG: Grounding Large Language Models to Holistic Segmentation
Most multimodal large language models (MLLMs) learn language-to-object grounding through causal language modeling where grounded objects are captured by bounding boxes as sequences of location tokens.
Ranked #1 on
Generalized Referring Expression Segmentation
on gRefCOCO
(using extra training data)
Causal Language Modeling
Generalized Referring Expression Segmentation
+2
Inversion-Free Image Editing with Natural Language
We show that when the initial sample is known, a special variance schedule reduces the denoising step to the same form as the multi-step consistency sampling.
Ranked #1 on
Text-based Image Editing
on PIE-Bench
Towards A Holistic Landscape of Situated Theory of Mind in Large Language Models
Such situated evaluation provides a more comprehensive assessment of mental states and potentially mitigates the risk of shortcuts and data leakage.
CycleNet: Rethinking Cycle Consistency in Text-Guided Diffusion for Image Manipulation
Our empirical studies show that Cyclenet is superior in translation consistency and quality, and can generate high-quality images for out-of-domain distributions with a simple change of the textual prompt.
World-to-Words: Grounded Open Vocabulary Acquisition through Fast Mapping in Vision-Language Models
The ability to connect language units to their referents in the physical world, referred to as grounding, is crucial to learning and understanding grounded meanings of words.
NLP Reproducibility For All: Understanding Experiences of Beginners
As natural language processing (NLP) has recently seen an unprecedented level of excitement, and more people are eager to enter the field, it is unclear whether current research reproducibility efforts are sufficient for this group of beginners to apply the latest developments.
Towards Collaborative Plan Acquisition through Theory of Mind Modeling in Situated Dialogue
To complete these tasks, agents need to engage in situated communication with their partners and coordinate their partial plans towards a complete plan to achieve a joint task goal.
DOROTHIE: Spoken Dialogue for Handling Unexpected Situations in Interactive Autonomous Driving Agents
To this end, we introduce Dialogue On the ROad To Handle Irregular Events (DOROTHIE), a novel interactive simulation platform that enables the creation of unexpected situations on the fly to support empirical studies on situated communication with autonomous driving agents.
DANLI: Deliberative Agent for Following Natural Language Instructions
These reactive agents are insufficient for long-horizon complex tasks.
Partition-Based Active Learning for Graph Neural Networks
We study the problem of semi-supervised learning with Graph Neural Networks (GNNs) in an active learning setup.
