Search Results for author:
Found 16 papers, 12 papers with code
MVBench: A Comprehensive Multi-modal Video Understanding Benchmark
With the rapid development of Multi-modal Large Language Models (MLLMs), a number of diagnostic benchmarks have recently emerged to evaluate the comprehension capabilities of these models.
Ranked #1 on
Zero-Shot Video Question Answer
on STAR Benchmark
Harvest Video Foundation Models via Efficient Post-Pretraining
Building video-language foundation models is costly and difficult due to the redundant nature of video data and the lack of high-quality video-language datasets.
InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation
Specifically, we utilize a multi-scale approach to generate video-related descriptions.
VideoChat: Chat-Centric Video Understanding
In this paper, we initiate an attempt of developing an end-to-end chat-centric video understanding system, coined as VideoChat.
Ranked #6 on
Video Question Answering
on MVBench
Video-based Generative Performance Benchmarking (Consistency)
Video-based Generative Performance Benchmarking (Contextual Understanding)
+5
Unmasked Teacher: Towards Training-Efficient Video Foundation Models
Previous VFMs rely on Image Foundation Models (IFMs), which face challenges in transferring to the video domain.
Ranked #1 on
Video Retrieval
on SSv2-template retrieval
(using extra training data)
UniFormerV2: Unlocking the Potential of Image ViTs for Video Understanding
The prolific performances of Vision Transformers (ViTs) in image tasks have prompted research into adapting the image ViTs for video tasks.
InternVideo: General Video Foundation Models via Generative and Discriminative Learning
Specifically, InternVideo efficiently explores masked video modeling and video-language contrastive learning as the pretraining objectives, and selectively coordinates video representations of these two complementary frameworks in a learnable manner to boost various video applications.
Ranked #1 on
Action Recognition
on Something-Something V1
(using extra training data)
UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormer
UniFormer has successfully alleviated this issue, by unifying convolution and self-attention as a relation aggregator in the transformer format.
InternVideo-Ego4D: A Pack of Champion Solutions to Ego4D Challenges
In this report, we present our champion solutions to five tracks at Ego4D challenge.
Ranked #1 on
State Change Object Detection
on Ego4D
HAKE: A Knowledge Engine Foundation for Human Activity Understanding
Human activity understanding is of widespread interest in artificial intelligence and spans diverse applications like health care and behavior analysis.
An Improved Reinforcement Learning Model Based on Sentiment Analysis
The authors combined the deep Q network in reinforcement learning with the sentiment quantitative indicator ARBR to build a high-frequency stock trading model for the share market.
Test-Time Personalization with a Transformer for Human Pose Estimation
During test time, we personalize and adapt our model by fine-tuning with the self-supervised objective.
PGT: A Progressive Method for Training Models on Long Videos
This progressive training (PGT) method is able to train long videos end-to-end with limited resources and ensures the effective transmission of information.
TDAF: Top-Down Attention Framework for Vision Tasks
Such spatial and attention features are nested deeply, therefore, the proposed framework works in a mixed top-down and bottom-up manner.
HOI Analysis: Integrating and Decomposing Human-Object Interaction
Meanwhile, isolated human and object can also be integrated into coherent HOI again.
Ranked #20 on
Human-Object Interaction Detection
on V-COCO
TubeTK: Adopting Tubes to Track Multi-Object in a One-Step Training Model
As deep learning brings excellent performances to object detection algorithms, Tracking by Detection (TBD) has become the mainstream tracking framework.
