Enhancing Multimodal Cooperation via Fine-grained Modality Valuation

1 code implementation • 12 Sep 2023 • Yake Wei, Ruoxuan Feng, Zihe Wang, Di Hu

One primary topic of multimodal learning is to jointly incorporate heterogeneous information from different modalities.

Revisiting Pre-training in Audio-Visual Learning

1 code implementation • 7 Feb 2023 • Ruoxuan Feng, Wenke Xia, Di Hu

Specifically, we explore the effects of pre-trained models on two audio-visual learning scenarios: cross-modal initialization and multi-modal joint learning.

audio-visual learning