Search Results for author:
Found 7 papers, 6 papers with code
Zero-shot audio captioning with audio-language model guidance and audio context keywords
In particular, our framework exploits a pre-trained large language model (LLM) for generating the text which is guided by a pre-trained audio-language model to produce captions that describe the audio content.
Ranked #1 on
Zero-shot Audio Captioning
on Clotho
Zero-shot Translation of Attention Patterns in VQA Models to Natural Language
Converting a model's internals to text can yield human-understandable insights about the model.
In-Context Impersonation Reveals Large Language Models' Strengths and Biases
These findings demonstrate that LLMs are capable of taking on diverse roles and that this in-context impersonation can be used to uncover their hidden strengths and biases.
Diverse Video Captioning by Adaptive Spatio-temporal Attention
To generate proper captions for videos, the inference needs to identify relevant concepts and pay attention to the spatial relationships between them as well as to the temporal development in the clip.
Ranked #7 on
Video Captioning
on VATEX
CLEVR-X: A Visual Reasoning Dataset for Natural Language Explanations
We present baseline results for generating natural language explanations in the context of VQA using two state-of-the-art frameworks on the CLEVR-X dataset.
Ranked #1 on
Explanation Generation
on CLEVR-X
e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Tasks
e-ViL is a benchmark for explainable vision-language tasks that establishes a unified evaluation framework and provides the first comprehensive comparison of existing approaches that generate NLEs for VL tasks.
Relational Generalized Few-Shot Learning
Instead, we consider the extended setup of generalized few-shot learning (GFSL), where the model is required to perform classification on the joint label space consisting of both previously seen and novel classes.
