Search Results for author:
Found 6 papers, 5 papers with code
Quantifying and Enhancing Multi-modal Robustness with Modality Preference
Moreover, our analysis reveals how the widespread issue, that the model has different preferences for modalities, limits the multi-modal robustness by influencing the essential components and could lead to attacks on the specific modality highly effective.
Enhancing Multimodal Cooperation via Fine-grained Modality Valuation
One primary topic of multimodal learning is to jointly incorporate heterogeneous information from different modalities.
Learning in Audio-visual Context: A Review, Analysis, and New Perspective
A comprehensive survey that can systematically organize and analyze studies of the audio-visual field is expected.
Balanced Multimodal Learning via On-the-fly Gradient Modulation
Multimodal learning helps to comprehensively understand the world, by integrating different senses.
Learning to Answer Questions in Dynamic Audio-Visual Scenarios
In this paper, we focus on the Audio-Visual Question Answering (AVQA) task, which aims to answer questions regarding different visual objects, sounds, and their associations in videos.
Ranked #5 on
Audio-visual Question Answering
on MUSIC-AVQA
Class-aware Sounding Objects Localization via Audiovisual Correspondence
To address this problem, we propose a two-stage step-by-step learning framework to localize and recognize sounding objects in complex audiovisual scenarios using only the correspondence between audio and vision.
