Search Results for author:
Found 61 papers, 42 papers with code
Unmasked Teacher: Towards Training-Efficient Video Foundation Models
Previous VFMs rely on Image Foundation Models (IFMs), which face challenges in transferring to the video domain.
CycleACR: Cycle Modeling of Actor-Context Relations for Video Action Detection
Existing studies model each actor and scene relation to improve action recognition.
LinK: Linear Kernel for LiDAR-based 3D Perception
Extending the success of 2D Large Kernel to 3D perception is challenging due to: 1. the cubically-increasing overhead in processing 3D data; 2. the optimization difficulties from data scarcity and sparsity.
PDPP:Projected Diffusion for Procedure Planning in Instructional Videos
In this sense, we model the whole intermediate action sequence distribution with a diffusion model (PDPP), and thus transform the planning problem to a sampling process from this distribution.
Learning Optical Flow and Scene Flow with Bidirectional Camera-LiDAR Fusion
To fuse dense image features and sparse point features, we propose a learnable operator named bidirectional camera-LiDAR fusion module (Bi-CLFM).
Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolation
In this paper, we propose a novel module to explicitly extract motion and appearance information via a unifying operation.
Ranked #1 on
Video Frame Interpolation
on UCF101
CoMAE: Single Model Hybrid Pre-training on Small-Scale RGB-D Datasets
Our CoMAE presents a curriculum learning strategy to unify the two popular self-supervised representation learning algorithms: contrastive learning and masked image modeling.
MixFormer: End-to-End Tracking with Iterative Mixed Attention
Our core design is to utilize the flexibility of attention operations, and propose a Mixed Attention Module (MAM) for simultaneous feature extraction and target information integration.
Ranked #1 on
Visual Object Tracking
on LaSOT
InternVideo: General Video Foundation Models via Generative and Discriminative Learning
Specifically, InternVideo efficiently explores masked video modeling and video-language contrastive learning as the pretraining objectives, and selectively coordinates video representations of these two complementary frameworks in a learnable manner to boost various video applications.
Ranked #1 on
Video Retrieval
on VATEX
VLG: General Video Recognition with Web Textual Knowledge
Our VLG is first pre-trained on video and language datasets to learn a shared feature space, and then devises a flexible bi-modal attention head to collaborate high-level semantic concepts under different settings.
UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormer
UniFormer has successfully alleviated this issue, by unifying convolution and self-attention as a relation aggregator in the transformer format.
InternVideo-Ego4D: A Pack of Champion Solutions to Ego4D Challenges
In this report, we present our champion solutions to five tracks at Ego4D challenge.
Ranked #1 on
State Change Object Detection
on Ego4D
Exploring State Change Capture of Heterogeneous Backbones @ Ego4D Hands and Objects Challenge 2022
Our method achieves an accuracy of 0. 796 on OSCC while achieving an absolute temporal localization error of 0. 516 on PNR.
Human-Object Interaction Detection
Object State Change Classification
+2
PointTAD: Multi-Label Temporal Action Detection with Learnable Query Points
Traditional temporal action detection (TAD) usually handles untrimmed videos with small number of action instances from a single label (e. g., ActivityNet, THUMOS).
Ranked #2 on
Temporal Action Localization
on MultiTHUMOS
Dynamic MDETR: A Dynamic Multimodal Transformer Decoder for Visual Grounding
The sampling module aims to select these informative patches by predicting the offsets with respect to a reference point, while the decoding module works for extracting the grounded object information by performing cross attention between image features and text features.
Submission to Generic Event Boundary Detection Challenge@CVPR 2022: Local Context Modeling and Global Boundary Decoding Approach
Local context modeling sub-network is proposed to perceive diverse patterns of generic event boundaries, and it generates powerful video representations and reliable boundary confidence.
Cross-Architecture Self-supervised Video Representation Learning
In this paper, we present a new cross-architecture contrastive learning (CACL) framework for self-supervised video representation learning.
BasicTAD: an Astounding RGB-Only Baseline for Temporal Action Detection
Empirical results demonstrate that our PlusTAD is very efficient and significantly outperforms the previous methods on the datasets of THUMOS14 and FineAction.
Ranked #1 on
Temporal Action Localization
on THUMOS14
APP-Net: Auxiliary-point-based Push and Pull Operations for Efficient Point Cloud Classification
In the existing work, each point in the cloud may inevitably be selected as the neighbors of multiple aggregation centers, as all centers will gather neighbor features from the whole point cloud independently.
Ranked #26 on
3D Point Cloud Classification
on ScanObjectNN
Joint-Modal Label Denoising for Weakly-Supervised Audio-Visual Video Parsing
This paper focuses on the weakly-supervised audio-visual video parsing task, which aims to recognize all events belonging to each modality and localize their temporal boundaries.
Logit Normalization for Long-tail Object Detection
Real-world data exhibiting skewed distributions pose a serious challenge to existing object detectors.
AdaMixer: A Fast-Converging Query-Based Object Detector
The recent query-based object detectors break this convention by decoding image features with a set of learnable queries.
Task-specific Inconsistency Alignment for Domain Adaptive Object Detection
To address this issue, in this paper, we propose Task-specific Inconsistency Alignment (TIA), by developing a new alignment mechanism in separate task spaces, improving the performance of the detector on both subtasks.
VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training
Pre-training video transformers on extra large-scale datasets is generally required to achieve premier performance on relatively small datasets.
Ranked #3 on
Self-Supervised Action Recognition
on UCF101
MixFormer: End-to-End Tracking with Iterative Mixed Attention
Our core design is to utilize the flexibility of attention operations, and propose a Mixed Attention Module (MAM) for simultaneous feature extraction and target information integration.
Ranked #4 on
Visual Object Tracking
on GOT-10k
Semi-Supervised Video Object Segmentation
Visual Object Tracking
Recovering 3D Human Mesh from Monocular Images: A Survey
Since the release of statistical body models, 3D human mesh recovery has been drawing broader attention.
Temporal Perceiver: A General Architecture for Arbitrary Boundary Detection
Instead, in this paper, we present Temporal Perceiver, a general architecture with Transformer, offering a unified solution to the detection of arbitrary generic boundaries, ranging from shot-level, event-level, to scene-level GBDs.
OCSampler: Compressing Videos to One Clip with Single-step Sampling
Recent works prefer to formulate frame sampling as a sequential decision task by selecting frames one by one according to their importance, while we present a new paradigm of learning instance-specific video condensation policies to select informative frames for representing the entire video only in a single step.
Progressive Attention on Multi-Level Dense Difference Maps for Generic Event Boundary Detection
Generic event boundary detection is an important yet challenging task in video understanding, which aims at detecting the moments where humans naturally perceive event boundaries.
DCAN: Improving Temporal Action Detection via Dual Context Aggregation
Specifically, we design the Multi-Path Temporal Context Aggregation (MTCA) to achieve smooth context aggregation on boundary level and precise evaluation of boundaries.
Ranked #12 on
Temporal Action Localization
on THUMOS’14
A Closer Look at Few-Shot Video Classification: A New Baseline and Benchmark
In this paper, we aim to present an in-depth study on few-shot video classification by making three contributions.
End-to-End Dense Video Grounding via Parallel Regression
In this paper, we tackle a new problem of dense video grounding, by simultaneously localizing multiple moments with a paragraph as input.
Mutual Supervision for Dense Object Detection
In this paper, we break the convention of the same training samples for these two heads in dense detectors and explore a novel supervisory paradigm, termed as Mutual Supervision (MuSu), to respectively and mutually assign training samples for the classification and regression head to ensure this consistency.
Negative Sample Matters: A Renaissance of Metric Learning for Temporal Grounding
Instead, from a perspective on temporal grounding as a metric-learning problem, we present a Mutual Matching Network (MMN), to directly model the similarity between language queries and video moments in a joint embedding space.
Self Supervision to Distillation for Long-Tailed Visual Recognition
In this paper, we show that soft label can serve as a powerful solution to incorporate label correlation into a multi-stage training scheme for long-tailed recognition.
Ranked #33 on
Long-tail Learning
on CIFAR-100-LT (ρ=100)
Target Adaptive Context Aggregation for Video Scene Graph Generation
Specifically, we design an efficient method for frame-level VidSGG, termed as {\em Target Adaptive Context Aggregation Network} (TRACE), with a focus on capturing spatio-temporal context information for relation recognition.
Structured Sparse R-CNN for Direct Scene Graph Generation
The key to our method is a set of learnable triplet queries and a structured triplet detector which could be jointly optimized from the training set in an end-to-end manner.
CGA-Net: Category Guided Aggregation for Point Cloud Semantic Segmentation
Previous point cloud semantic segmentation networks use the same process to aggregate features from neighbors of the same category and different categories.
Ranked #1 on
Semantic Segmentation
on SYNTHIA
Joint Landmark and Structure Learning for Automatic Evaluation of Developmental Dysplasia of the Hip
In this study, we propose a multi-task framework to learn the relationships among landmarks and structures jointly and automatically evaluate DDH.
SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction
Three-dimensional face dense alignment and reconstruction in the wild is a challenging problem as partial facial information is commonly missing in occluded and large pose face images.
Ranked #1 on
3D Face Reconstruction
on AFLW2000-3D
FineAction: A Fine-Grained Video Dataset for Temporal Action Localization
Temporal action localization (TAL) is an important and challenging problem in video understanding.
Fine-Grained Action Detection
Temporal Action Localization
+2
MultiSports: A Multi-Person Video Dataset of Spatio-Temporally Localized Sports Actions
Spatio-temporal action detection is an important and challenging problem in video understanding.
MGSampler: An Explainable Sampling Strategy for Video Action Recognition
First, we present two different motion representations to enable us to efficiently distinguish the motion-salient frames from the background.
Target Transformed Regression for Accurate Tracking
Accurate tracking is still a challenging task due to appearance variations, pose and view changes, and geometric deformations of target in videos.
Ranked #1 on
Visual Object Tracking
on VOT2019
PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop
Regression-based methods have recently shown promising results in reconstructing human meshes from monocular images.
Ranked #32 on
3D Human Pose Estimation
on 3DPW
(using extra training data)
3D human pose and shape estimation
3D Human Reconstruction
+2
Relaxed Transformer Decoders for Direct Action Proposal Generation
Extensive experiments on THUMOS14 and ActivityNet-1. 3 benchmarks demonstrate the effectiveness of RTD-Net, on both tasks of temporal action proposal generation and temporal action detection.
Temporal Difference Networks for Action Recognition
To mitigate this issue, this paper presents a new video architecture, termed as Temporal Difference Network (TDN), with a focus on capturing multi-scale temporal information for efficient action recognition.
TDN: Temporal Difference Networks for Efficient Action Recognition
To mitigate this issue, this paper presents a new video architecture, termed as Temporal Difference Network (TDN), with a focus on capturing multi-scale temporal information for efficient action recognition.
Ranked #11 on
Action Recognition
on Something-Something V1
Appearance-and-Relation Networks for Video Classification
Specifically, SMART blocks decouple the spatiotemporal learning module into an appearance branch for spatial modeling and a relation branch for temporal modeling.
Ranked #47 on
Action Recognition
on UCF101
Temporal Segment Networks for Action Recognition in Videos
Furthermore, based on the temporal segment networks, we won the video classification track at the ActivityNet challenge 2016 among 24 teams, which demonstrates the effectiveness of TSN and the proposed good practices.
Ranked #18 on
Action Classification
on Moments in Time
(Top 5 Accuracy metric)
UntrimmedNets for Weakly Supervised Action Recognition and Detection
We exploit the learned models for action recognition (WSR) and detection (WSD) on the untrimmed video datasets of THUMOS14 and ActivityNet.
Ranked #3 on
Action Classification
on THUMOS’14
Weakly Supervised Action Localization
Weakly-Supervised Action Recognition
Knowledge Guided Disambiguation for Large-Scale Scene Classification with Multi-Resolution CNNs
Convolutional Neural Networks (CNNs) have made remarkable progress on scene recognition, partially due to these recent large-scale scene datasets, such as the Places and Places2.
Transferring Object-Scene Convolutional Neural Networks for Event Recognition in Still Images
These newly designed transferring techniques exploit multi-task learning frameworks to incorporate extra knowledge from other networks and additional datasets into the training procedure of event CNNs.
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
The other contribution is our study on a series of good practices in learning ConvNets on video data with the help of temporal segment network.
Ranked #3 on
Multimodal Activity Recognition
on EV-Action
Actionness Estimation Using Hybrid Fully Convolutional Networks
Actionness was introduced to quantify the likelihood of containing a generic action instance at a specific location.
Ranked #7 on
Temporal Action Localization
on J-HMDB-21
Better Exploiting OS-CNNs for Better Event Recognition in Images
Event recognition from still images is one of the most important problems for image understanding.
Places205-VGGNet Models for Scene Recognition
We verify the performance of trained Places205-VGGNet models on three datasets: MIT67, SUN397, and Places205.
Towards Good Practices for Very Deep Two-Stream ConvNets
However, for action recognition in videos, the improvement of deep convolutional networks is not so evident.
Ranked #62 on
Action Recognition
on UCF101
Action Recognition with Trajectory-Pooled Deep-Convolutional Descriptors
Visual features are of vital importance for human action understanding in videos.
Ranked #2 on
Activity Recognition In Videos
on DogCentric
Object-Scene Convolutional Neural Networks for Event Recognition in Images
Meanwhile, we investigate different network architectures for OS-CNN design, and adapt the deep (AlexNet) and very-deep (GoogLeNet) networks to the task of event recognition.
