Video-Text Retrieval
56 papers with code • 1 benchmarks • 6 datasets
Video-Text retrieval requires understanding of both video and language together. Therefore it's different to video retrieval task.
Benchmarks
These leaderboards are used to track progress in Video-Text Retrieval
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
Libraries
Use these libraries to find Video-Text Retrieval models and implementations
WebVid
Test-of-Time
SYMON
Youku-mPLUG
VTC
Most implemented papers
LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment
pku-yuangroup/languagebind
•
•
We thus propose VIDAL-10M with Video, Infrared, Depth, Audio and their corresponding Language, naming as VIDAL-10M.
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval
m-bain/frozen-in-time
•
•
ICCV 2021
Our objective in this work is video-text retrieval - in particular a joint embedding that enables efficient text-to-video retrieval.
CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval
In this paper, we propose a CLIP4Clip model to transfer the knowledge of the CLIP model to video-language retrieval in an end-to-end manner.
Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding
We present Video-LLaMA a multi-modal framework that empowers Large Language Models (LLMs) with the capability of understanding both visual and auditory content in the video.
Fine-grained Video-Text Retrieval with Hierarchical Graph Reasoning
CryhanFang/CLIP2Video
•
•
CVPR 2020
To improve fine-grained video-text retrieval, we propose a Hierarchical Graph Reasoning (HGR) model, which decomposes video-text matching into global-to-local levels.
mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections
Large-scale pretrained foundation models have been an emerging paradigm for building artificial intelligence (AI) systems, which can be quickly adapted to a wide range of downstream tasks.
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval
xuguohai/X-CLIP
•
•
However, cross-grained contrast, which is the contrast between coarse-grained representations and fine-grained representations, has rarely been explored in prior research.
Improving Video-Text Retrieval by Multi-Stream Corpus Alignment and Dual Softmax Loss
starmemda/camow
•
•
In this paper, we propose a multi-stream Corpus Alignment network with single gate Mixture-of-Experts (CAMoE) and a novel Dual Softmax Loss (DSL) to solve the two heterogeneity.
Bridging Video-text Retrieval with Multiple Choice Questions
As an additional benefit, our method achieves competitive results with much shorter pre-training videos on single-modality downstream tasks, e. g., action recognition with linear evaluation.
Egocentric Video-Language Pretraining
showlab/egovlp
•
•
Video-Language Pretraining (VLP), which aims to learn transferable representation to advance a wide range of video-text downstream tasks, has recently received increasing attention.
