Search Results for author:
Found 11 papers, 7 papers with code
Vision-Language Instruction Tuning: A Review and Analysis
Instruction tuning is a crucial supervised training phase in Large Language Models (LLMs), aiming to enhance the LLM's ability to generalize instruction execution and adapt to user preferences.
TagGPT: Large Language Models are Zero-shot Multimodal Taggers
Given a new entity that needs tagging for distribution, TagGPT introduces two alternative options for zero-shot tagging, i. e., a generative method with late semantic matching with the tag set, and another selective method with early matching in prompts.
RILS: Masked Visual Reconstruction in Language Semantic Space
Both masked image modeling (MIM) and natural language supervision have facilitated the progress of transferable visual pre-training.
Masked Image Modeling with Denoising Contrast
Since the development of self-supervised visual representation learning from contrastive learning to masked image modeling (MIM), there is no significant difference in essence, that is, how to design proper pretext tasks for vision dictionary look-up.
Controllable Augmentations for Video Representation Learning
The major reason is that the positive pairs, i. e., different clips sampled from the same video, have limited temporal receptive field, and usually share similar background but differ in motions.
Bridging Video-text Retrieval with Multiple Choice Questions
As an additional benefit, our method achieves competitive results with much shorter pre-training videos on single-modality downstream tasks, e. g., action recognition with linear evaluation.
Ranked #8 on
Zero-Shot Video Retrieval
on MSVD
Tencent-MVSE: A Large-Scale Benchmark Dataset for Multi-Modal Video Similarity Evaluation
In this paper, we propose the Tencent-MVSE dataset, which is the first benchmark dataset for the multi-modal video similarity evaluation task.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
CA-SSL: Class-Agnostic Semi-Supervised Learning for Detection and Segmentation
To improve instance-level detection/segmentation performance, existing self-supervised and semi-supervised methods extract either task-unrelated or task-specific training signals from unlabeled data.
CLIP4Caption: CLIP for Video Caption
It is noted that our model is only trained on the MSR-VTT dataset.
CLIP4Caption ++: Multi-CLIP for Video Caption
We make the following improvements on the proposed CLIP4Caption++: We employ an advanced encoder-decoder model architecture X-Transformer as our main framework and make the following improvements: 1) we utilize three strong pre-trained CLIP models to extract the text-related appearance visual features.
Enhancing Self-supervised Video Representation Learning via Multi-level Feature Optimization
The crux of self-supervised video representation learning is to build general features from unlabeled videos.
