Search Results for author:
Found 9 papers, 7 papers with code
ViUniT: Visual Unit Tests for More Robust Visual Programming
In our framework, a unit test is represented as a novel image and answer pair meant to verify the logical correctness of a program produced for a given query.
Evaluating Vision-Language Models on Bistable Images
Bistable images, also known as ambiguous or reversible images, present visual stimuli that can be seen in two distinct interpretations, though not simultaneously by the observer.
X-InstructBLIP: A Framework for aligning X-Modal instruction-aware representations to LLMs and Emergent Cross-modal Reasoning
To enable this framework, we devise a scalable pipeline that automatically generates high-quality, instruction-tuning datasets from readily available captioning data across different modalities, and contribute 24K QA data for audio and 250K QA data for 3D.
I Spy a Metaphor: Large Language Models and Diffusion Models Co-Create Visual Metaphors
We propose to solve the task through the collaboration between Large Language Models (LLMs) and Diffusion Models: Instruct GPT-3 (davinci-002) with Chain-of-Thought prompting generates text that represents a visual elaboration of the linguistic metaphor containing the implicit meaning and relevant objects, which is then used as input to the diffusion-based text-to-image models. Using a human-AI collaboration framework, where humans interact both with the LLM and the top-performing diffusion model, we create a high-quality dataset containing 6, 476 visual metaphors for 1, 540 linguistic metaphors and their associated visual elaborations.
ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding
It achieves a new SOTA of 50. 6% (top-1) on Objaverse-LVIS and 84. 7% (top-1) on ModelNet40 in zero-shot classification.
Ranked #10 on
3D Point Cloud Classification
on ScanObjectNN
(using extra training data)
Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification
Overall, LaBo demonstrates that inherently interpretable models can be widely applied at similar, or better, performance than black box approaches.
Visualizing the Obvious: A Concreteness-based Ensemble Model for Noun Property Prediction
We propose to extract these properties from images and use them in an ensemble model, in order to complement the information that is extracted from language models.
Induce, Edit, Retrieve:Language Grounded Multimodal Schema for Instructional Video Retrieval
Schemata are structured representations of complex tasks that can aid artificial intelligence by allowing models to break down complex tasks into intermediate steps.
Visual Goal-Step Inference using wikiHow
Understanding what sequence of steps are needed to complete a goal can help artificial intelligence systems reason about human activities.
Ranked #1 on
VGSI
on wikiHow-image
