Zero-Shot Video Retrieval
30 papers with code • 8 benchmarks • 7 datasets
Benchmarks
These leaderboards are used to track progress in Zero-Shot Video Retrieval
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Zero-Shot Video Retrieval models and implementations
ActivityNet
MSR-VTT
MSVD
DiDeMo
YouCook2
LSMDC
VATEX
Most implemented papers
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval
m-bain/frozen-in-time
•
•
ICCV 2021
Our objective in this work is video-text retrieval - in particular a joint embedding that enables efficient text-to-video retrieval.
CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval
In this paper, we propose a CLIP4Clip model to transfer the knowledge of the CLIP model to video-language retrieval in an end-to-end manner.
End-to-End Learning of Visual Representations from Uncurated Instructional Videos
antoine77340/MIL-NCE_HowTo100M
•
•
CVPR 2020
Annotating videos is cumbersome, expensive and not scalable.
mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video
In contrast to predominant paradigms of solely relying on sequence-to-sequence generation or encoder-based instance discrimination, mPLUG-2 introduces a multi-module composition network by sharing common universal modules for modality collaboration and disentangling different modality modules to deal with modality entanglement.
LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment
pku-yuangroup/languagebind
•
•
We thus propose VIDAL-10M with Video, Infrared, Depth, Audio and their corresponding Language, naming as VIDAL-10M.
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text
google-research/google-research
•
•
NeurIPS 2021
We train VATT end-to-end from scratch using multimodal contrastive losses and evaluate its performance by the downstream tasks of video action recognition, audio event classification, image classification, and text-to-video retrieval.
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding
pytorch/fairseq
•
•
EMNLP 2021
We present VideoCLIP, a contrastive approach to pre-train a unified model for zero-shot video and text understanding, without using any labels on downstream tasks.
Bridging Video-text Retrieval with Multiple Choice Questions
As an additional benefit, our method achieves competitive results with much shorter pre-training videos on single-modality downstream tasks, e. g., action recognition with linear evaluation.
Revealing Single Frame Bias for Video-and-Language Learning
Training an effective video-and-language model intuitively requires multiple frames as model inputs.
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
However, the progress in vision and vision-language foundation models, which are also critical elements of multi-modal AGI, has not kept pace with LLMs.
