Search Results for author:
Found 12 papers, 8 papers with code
PlanT: Explainable Planning Transformers via Object-Level Representations
Planning an optimal route in a complex environment requires efficient reasoning about the surrounding scene.
Temporal and cross-modal attention for audio-visual zero-shot learning
We show that our proposed framework that ingests temporal features yields state-of-the-art performance on the \ucf, \vgg, and \activity benchmarks for (generalised) zero-shot learning.
CLEVR-X: A Visual Reasoning Dataset for Natural Language Explanations
We present baseline results for generating natural language explanations in the context of VQA using two state-of-the-art frameworks on the CLEVR-X dataset.
Ranked #1 on
Explanation Generation
on CLEVR-X
Audio-visual Generalised Zero-shot Learning with Cross-modal Attention and Language
Focusing on the relatively underexplored task of audio-visual zero-shot learning, we propose to learn multi-modal representations from audio-visual data using cross-modal attention and exploit textual label embeddings for transferring knowledge from seen classes to unseen classes.
Ranked #1 on
GZSL Video Classification
on VGGSound-GZSL (cls)
Audio Retrieval with Natural Language Queries: A Benchmark Study
Additionally, we introduce the SoundDescs benchmark, which consists of paired audio and natural language descriptions for a diverse collection of sounds that are complementary to those found in AudioCaps and Clotho.
Ranked #1 on
Text to Audio Retrieval
on AudioCaps
Audio Retrieval with Natural Language Queries
We consider the task of retrieving audio using free-form natural language queries.
Ranked #1 on
Audio to Text Retrieval
on Clotho
Where and When: Space-Time Attention for Audio-Visual Explanations
Recent advances in XAI provide explanations for models trained on still images.
Explainable artificial intelligence
Multimodal Deep Learning
Distilling Audio-Visual Knowledge by Compositional Contrastive Learning
Having access to multi-modal cues (e. g. vision and audio) empowers some cognitive tasks to be done faster compared to learning from a single modality.
Self-supervised learning of class embeddings from video
This work explores how to use self-supervised learning on videos to learn a class-specific image embedding that encodes pose and shape information.
X2Face: A network for controlling face generation using images, audio, and pose codes
The objective of this paper is a neural network model that controls the pose and expression of a given face, using another face or modality (e. g. audio).
Self-supervised learning of a facial attribute embedding from video
We propose a self-supervised framework for learning facial attributes by simply watching videos of a human face speaking, laughing, and moving over time.
Ranked #2 on
Unsupervised Facial Landmark Detection
on 300W
Self-Supervised Learning
Unsupervised Facial Landmark Detection
X2Face: A network for controlling face generation by using images, audio, and pose codes
The objective of this paper is a neural network model that controls the pose and expression of a given face, using another face or modality (e. g. audio).
