Search Results for author:
Found 10 papers, 6 papers with code
Siamese Vision Transformers are Scalable Audio-visual Learners
Our framework uses a single shared vision transformer backbone to process audio and visual inputs, improving its parameter efficiency, reducing the GPU memory footprint, and allowing us to scale our method to larger datasets and model sizes.
DAM: Dynamic Adapter Merging for Continual Video QA Learning
Our DAM model outperforms prior state-of-the-art continual learning approaches by 9. 1% while exhibiting 1. 9% less forgetting on 6 VidQA datasets spanning various domains.
Vision Transformers are Parameter-Efficient Audio-Visual Learners
To do so, we propose a latent audio-visual hybrid (LAVISH) adapter that adapts pretrained ViTs to audio-visual tasks by injecting a small number of trainable parameters into every layer of a frozen ViT.
Ranked #4 on
Audio-visual Question Answering
on MUSIC-AVQA
ECLIPSE: Efficient Long-range Video Retrieval using Sight and Sound
We introduce an audiovisual method for long-range text-to-video retrieval.
Exploring Cross-Video and Cross-Modality Signals for Weakly-Supervised Audio-Visual Video Parsing
The audio-visual video parsing task aims to temporally parse a video into audio or visual event categories.
Exploiting Audio-Visual Consistency with Partial Supervision for Spatial Audio Generation
Human perceives rich auditory experience with distinct sound heard by ears.
Unsupervised Sound Localization via Iterative Contrastive Learning
Sound localization aims to find the source of the audio signal in the visual scene.
Cross-Dataset Person Re-Identification via Unsupervised Pose Disentanglement and Adaptation
Person re-identification (re-ID) aims at recognizing the same person from images taken across different cameras.
Ranked #16 on
Unsupervised Domain Adaptation
on Market to Duke
Dual-modality seq2seq network for audio-visual event localization
Audio-visual event localization requires one to identify theevent which is both visible and audible in a video (eitherat a frame or video level).
