Search Results for author:
Found 15 papers, 5 papers with code
Audio Dialogues: Dialogues dataset for audio and music understanding
Existing datasets for audio understanding primarily focus on single-turn interactions (i. e. audio captioning, audio question answering) for describing audio in natural language, thus limiting understanding audio via interactive dialogue.
Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities
Augmenting large language models (LLMs) to understand audio -- including non-speech sounds and non-verbal speech -- is critically important for diverse real-world applications of LLMs.
Language-guided Robot Grasping: CLIP-based Referring Grasp Synthesis in Clutter
To address these limitations, we develop a challenging benchmark based on cluttered indoor scenes from OCID dataset, for which we generate referring expressions and connect them with 4-DoF grasp poses.
Semi-supervised multimodal coreference resolution in image narrations
In this paper, we study multimodal coreference resolution, specifically where a longer descriptive text, i. e., a narration is paired with an image.
Encyclopedic VQA: Visual questions about detailed properties of fine-grained categories
Empirically, we show that our dataset poses a hard challenge for large vision+language models as they perform poorly on our dataset: PaLI [14] is state-of-the-art on OK-VQA [37], yet it only achieves 13. 0% accuracy on our dataset.
Controllable Video Generation by Learning the Underlying Dynamical System with Neural ODE
Videos depict the change of complex dynamical systems over time in the form of discrete image sequences.
Who are you referring to? Coreference resolution in image narrations
Coreference resolution aims to identify words and phrases which refer to same entity in a text, a core task in natural language processing.
WiCV 2022: The Tenth Women In Computer Vision Workshop
In this paper, we present the details of Women in Computer Vision Workshop - WiCV 2022, organized alongside the hybrid CVPR 2022 in New Orleans, Louisiana.
WiCV 2021: The Eighth Women In Computer Vision Workshop
In this paper, we present the details of Women in Computer Vision Workshop - WiCV 2021, organized alongside the virtual CVPR 2021.
PARS: Pseudo-Label Aware Robust Sample Selection for Learning with Noisy Labels
In this paper, we propose PARS: Pseudo-Label Aware Robust Sample Selection, a hybrid approach that combines the best from all three worlds in a joint-training framework to achieve robustness to noisy labels.
Not All Relations are Equal: Mining Informative Labels for Scene Graph Generation
Scene graph generation (SGG) aims to capture a wide variety of interactions between pairs of objects, which is essential for full scene understanding.
Injecting Prior Knowledge into Image Caption Generation
Automatically generating natural language descriptions from an image is a challenging problem in artificial intelligence that requires a good understanding of the visual and textual signals and the correlations between them.
Cross-Domain Image Classification through Neural-Style Transfer Data Augmentation
In particular, the lack of sufficient amounts of domain-specific data can reduce the accuracy of a classifier.
An End-to-End Network for Generating Social Relationship Graphs
Inferring the social context in a given visual scene not only involves recognizing objects, but also demands a more in-depth understanding of the relationships and attributes of the people involved.
A Multimodal LSTM for Predicting Listener Empathic Responses Over Time
People naturally understand the emotions of-and often also empathize with-those around them.
