Search Results for author:
Found 13 papers, 10 papers with code
COSMO: COntrastive Streamlined MultimOdal Model with Interleaved Pre-Training
\ModelName, our unified framework, merges unimodal and multimodal elements, enhancing model performance for tasks involving textual and visual data while notably reducing learnable parameters.
Bootstrapping SparseFormers from Vision Foundation Models
In this paper, we propose to bootstrap SparseFormers from ViT-based vision foundation models in a simple and efficient way.
DiffusionVMR: Diffusion Model for Joint Video Moment Retrieval and Highlight Detection
An arbitrary setting can be used in DiffusionVMR during inference without consistency with the training phase.
Ranked #4 on
Video Grounding
on QVHighlights
UniVTG: Towards Unified Video-Language Temporal Grounding
Most methods in this direction develop taskspecific models that are trained with type-specific labels, such as moment retrieval (time interval) and highlight detection (worthiness curve), which limits their abilities to generalize to various VTG tasks and labels.
Ranked #3 on
Natural Language Moment Retrieval
on TACoS
EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbone
Video-language pre-training (VLP) has become increasingly important due to its ability to generalize to various vision and language tasks.
AssistGPT: A General Multi-modal Assistant that can Plan, Execute, Inspect, and Learn
2) Flexible inputs and intermediate results.
Too Large; Data Reduction for Vision-Language Pre-Training
Specifically, TL;DR can compress the mainstream VLP datasets at a high ratio, e. g., reduce well-cleaned CC3M dataset from 2. 82M to 0. 67M ($\sim$24\%) and noisy YFCC15M from 15M to 2. 5M ($\sim$16. 7\%).
VisorGPT: Learning Visual Prior via Generative Pre-Training
Experimental results demonstrate that VisorGPT can effectively model the visual prior, which can be employed for many vision tasks, such as customizing accurate human pose for conditional image synthesis models like ControlNet.
Affordance Grounding from Demonstration Video to Target Image
Humans excel at learning from expert demonstrations and solving their own problems.
Ranked #1 on
Video-to-image Affordance Grounding
on EPIC-Hotspot
Unsupervised Hashing with Semantic Concept Mining
Recently, to improve the unsupervised image retrieval performance, plenty of unsupervised hashing methods have been proposed by designing a semantic similarity matrix, which is based on the similarities between image features extracted by a pre-trained CNN model.
Egocentric Video-Language Pretraining @ Ego4D Challenge 2022
In this report, we propose a video-language pretraining (VLP) based solution \cite{kevin2022egovlp} for four Ego4D challenge tasks, including Natural Language Query (NLQ), Moment Query (MQ), Object State Change Classification (OSCC), and PNR Localization (PNR).
Egocentric Video-Language Pretraining @ EPIC-KITCHENS-100 Multi-Instance Retrieval Challenge 2022
In this report, we propose a video-language pretraining (VLP) based solution \cite{kevin2022egovlp} for the EPIC-KITCHENS-100 Multi-Instance Retrieval (MIR) challenge.
Egocentric Video-Language Pretraining
Video-Language Pretraining (VLP), which aims to learn transferable representation to advance a wide range of video-text downstream tasks, has recently received increasing attention.
