Search Results for author:
Found 32 papers, 24 papers with code
Betrayed by Attention: A Simple yet Effective Approach for Self-supervised Video Object Segmentation
In this paper, we propose a simple yet effective approach for self-supervised video object segmentation (VOS).
Semantics Meets Temporal Correspondence: Self-supervised Object-centric Learning in Videos
In the second stage, for each semantics, we randomly sample slots from the corresponding Gaussian distribution and perform masked feature aggregation within the semantic area to exploit temporal correspondence patterns for instance identification.
Prune Spatio-temporal Tokens by Semantic-aware Temporal Accumulation
Based on the STA score, we are able to progressively prune the tokens without introducing any additional parameters or requiring further re-training.
Taming Diffusion Models for Audio-Driven Co-Speech Gesture Generation
In this work, we propose a novel diffusion-based framework, named Diffusion Co-Speech Gesture (DiffGesture), to effectively capture the cross-modal audio-to-gesture associations and preserve temporal coherence for high-fidelity audio-driven co-speech gesture generation.
Motion-inductive Self-supervised Object Discovery in Videos
In this paper, we consider the task of unsupervised object discovery in videos.
Ranked #3 on
Unsupervised Object Segmentation
on DAVIS 2016
Static and Dynamic Concepts for Self-supervised Video Representation Learning
In this paper, we propose a novel learning scheme for self-supervised video representation learning.
Exploring Fine-Grained Audiovisual Categorization with the SSW60 Dataset
We thoroughly benchmark audiovisual classification performance and modality fusion experiments through the use of state-of-the-art transformer methods.
Multimodal Open-Vocabulary Video Classification via Pre-Trained Vision and Language Models
Utilizing vision and language models (VLMs) pre-trained on large-scale image-text pairs is becoming a promising paradigm for open-vocabulary visual recognition.
Ranked #1 on
Zero-Shot Action Recognition
on HMDB51
Dual Contrastive Learning for Spatio-temporal Representation
In this way, the static scene and the dynamic motion are simultaneously encoded into the compact RGB representation.
Controllable Augmentations for Video Representation Learning
The major reason is that the positive pairs, i. e., different clips sampled from the same video, have limited temporal receptive field, and usually share similar background but differ in motions.
Learning Hierarchical Cross-Modal Association for Co-Speech Gesture Generation
To enhance the quality of synthesized gestures, we develop a contrastive learning strategy based on audio-text alignment for better audio representations.
Ranked #3 on
Gesture Generation
on TED Gesture Dataset
Visual Sound Localization in the Wild by Cross-Modal Interference Erasing
Specifically, we observe that the previous practice of learning only a single audio representation is insufficient due to the additive nature of audio signals.
Class-aware Sounding Objects Localization via Audiovisual Correspondence
To address this problem, we propose a two-stage step-by-step learning framework to localize and recognize sounding objects in complex audiovisual scenarios using only the correspondence between audio and vision.
Contextualized Spatio-Temporal Contrastive Learning with Self-Supervision
Modern self-supervised learning algorithms typically enforce persistency of instance representations across views.
Exploring Temporal Granularity in Self-Supervised Video Representation Learning
The training objective consists of two parts: a fine-grained temporal learning objective to maximize the similarity between corresponding temporal embeddings in the short clip and the long clip, and a persistent temporal learning objective to pull together global embeddings of the two clips.
Motion-aware Contrastive Video Representation Learning via Foreground-background Merging
To alleviate such bias, we propose \textbf{F}oreground-b\textbf{a}ckground \textbf{Me}rging (FAME) to deliberately compose the moving foreground region of the selected video onto the static background of others.
Revisiting 3D ResNets for Video Recognition
A recent work from Bello shows that training and scaling strategies may be more significant than model architectures for visual recognition.
Ranked #38 on
Action Classification
on Kinetics-600
Enhancing Self-supervised Video Representation Learning via Multi-level Feature Optimization
The crux of self-supervised video representation learning is to build general features from unlabeled videos.
TA2N: Two-Stage Action Alignment Network for Few-shot Action Recognition
The first stage locates the action by learning a temporal affine transform, which warps each video feature to its action duration while dismissing the action-irrelevant feature (e. g. background).
3D Object Detection for Autonomous Driving: A Survey
Autonomous driving is regarded as one of the most promising remedies to shield human beings from severe crashes.
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text
We train VATT end-to-end from scratch using multimodal contrastive losses and evaluate its performance by the downstream tasks of video action recognition, audio event classification, image classification, and text-to-video retrieval.
Ranked #9 on
Action Classification
on Moments in Time
(using extra training data)
BADet: Boundary-Aware 3D Object Detection from Point Clouds
Specifically, instead of refining each proposal independently as previous works do, we represent each proposal as a node for graph construction within a given cut-off threshold, associating proposals in the form of local neighborhood graph, with boundary correlations of an object being explicitly exploited.
Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation
Our baseline model outperforms the LVIS 2020 Challenge winning entry by +3. 6 mask AP on rare categories.
Ranked #1 on
Object Detection
on PASCAL VOC 2007
Discriminative Sounding Objects Localization via Self-supervised Audiovisual Matching
First, we propose to learn robust object representations by aggregating the candidate sound localization results in the single source scenes.
Finding Action Tubes with a Sparse-to-Dense Framework
The task of spatial-temporal action detection has attracted increasing attention among researchers.
Ranked #3 on
Action Detection
on UCF Sports
(Video-mAP 0.2 metric)
Spatiotemporal Contrastive Video Representation Learning
Our representations are learned using a contrastive loss, where two augmented clips from the same short video are pulled together in the embedding space, while clips from different videos are pushed away.
Ranked #1 on
Self-Supervised Action Recognition
on Kinetics-600
Multiple Sound Sources Localization from Coarse to Fine
How to visually localize multiple sound sources in unconstrained videos is a formidable problem, especially when lack of the pairwise sound-object annotations.
Human in Events: A Large-Scale Benchmark for Human-centric Video Analysis in Complex Events
To this end, we present a new large-scale dataset with comprehensive annotations, named Human-in-Events or HiEve (Human-centric video analysis in complex Events), for the understanding of human motions, poses, and actions in a variety of realistic events, especially in crowd & complex events.
End-to-End Pseudo-LiDAR for Image-Based 3D Object Detection
Reliable and accurate 3D object detection is a necessity for safe autonomous driving.
ATRW: A Benchmark for Amur Tiger Re-identification in the Wild
This paper tries to fill the gap by introducing a novel large-scale dataset, the Amur Tiger Re-identification in the Wild (ATRW) dataset.
Weakly Supervised Scene Parsing with Point-based Distance Metric Learning
Semantic scene parsing is suffering from the fact that pixel-level annotations are hard to be collected.
Attentive Generative Adversarial Network for Raindrop Removal from a Single Image
This injection of visual attention to both generative and discriminative networks is the main contribution of this paper.
