Search Results for author:
Found 18 papers, 13 papers with code
Contextual AD Narration with Interleaved Multimodal Sequence
With video feature, text, character bank and context information as inputs, the generated ADs are able to correspond to the characters by name and provide reasonable, contextual descriptions to help audience understand the storyline of movie.
TagAlign: Improving Vision-Language Alignment with Multi-Tag Classification
The crux of learning vision-language models is to extract semantically aligned information from visual and linguistic data.
Bootstrapping SparseFormers from Vision Foundation Models
In this paper, we propose to bootstrap SparseFormers from ViT-based vision foundation models in a simple and efficient way.
Advancing Vision Transformers with Group-Mix Attention
The attention map is computed based on the mixtures of tokens and group proxies and used to re-combine the tokens and groups in Value.
Speed Co-Augmentation for Unsupervised Audio-Visual Pre-training
This work aims to improve unsupervised audio-visual pre-training.
TVTSv2: Learning Out-of-the-box Spatiotemporal Visual Representations at Scale
We argue that tuning a text encoder end-to-end, as done in previous work, is suboptimal since it may overfit in terms of styles, thereby losing its original generalization ability to capture the semantics of various language registers.
Efficient Video Action Detection with Token Dropout and Context Refinement
Our EVAD consists of two specialized designs for video action detection.
SparseFormer: Sparse Visual Recognition via Limited Latent Tokens
In this paper, we challenge this dense paradigm and present a new method, coined SparseFormer, to imitate human's sparse visual recognition in an end-to-end manner.
Sparse Representation-based Classification
Video Classification
Soft Neighbors are Positive Supporters in Contrastive Visual Representation Learning
We evaluate our soft neighbor contrastive learning method (SNCLR) on standard visual recognition benchmarks, including image classification, object detection, and instance segmentation.
VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking
Finally, we successfully train a video ViT model with a billion parameters, which achieves a new state-of-the-art performance on the datasets of Kinetics (90. 0% on K400 and 89. 9% on K600) and Something-Something (68. 7% on V1 and 77. 0% on V2).
Ranked #1 on
Self-Supervised Action Recognition
on UCF101
(using extra training data)
CycleACR: Cycle Modeling of Actor-Context Relations for Video Action Detection
Existing studies model each actor and scene relation to improve action recognition.
AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition
To address this challenge, we propose an effective adaptation approach for Transformer, namely AdaptFormer, which can adapt the pre-trained ViTs into many different image and video tasks efficiently.
VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training
Pre-training video transformers on extra large-scale datasets is generally required to achieve premier performance on relatively small datasets.
Ranked #3 on
Self-Supervised Action Recognition Linear
on HMDB51
Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizations
Second, by maintaining the same computational cost, our method empowers ViTs to take more image tokens as input for recognition accuracy improvement, where the image tokens are from higher resolution images.
Ranked #4 on
Efficient ViTs
on ImageNet-1K (with DeiT-S)
EViT: Expediting Vision Transformers via Token Reorganizations
Second, by maintaining the same computational cost, our method empowers ViTs to take more image tokens as input for recognition accuracy improvement, where the image tokens are from higher resolution images.
MGSampler: An Explainable Sampling Strategy for Video Action Recognition
First, we present two different motion representations to enable us to efficiently distinguish the motion-salient frames from the background.
Temporal Difference Networks for Action Recognition
To mitigate this issue, this paper presents a new video architecture, termed as Temporal Difference Network (TDN), with a focus on capturing multi-scale temporal information for efficient action recognition.
TDN: Temporal Difference Networks for Efficient Action Recognition
To mitigate this issue, this paper presents a new video architecture, termed as Temporal Difference Network (TDN), with a focus on capturing multi-scale temporal information for efficient action recognition.
Ranked #18 on
Action Recognition
on Something-Something V1
