Search Results for author:
Found 40 papers, 17 papers with code
Learning Spatial Adaptation and Temporal Coherence in Diffusion Models for Video Super-Resolution
Technically, SATeCo freezes all the parameters of the pre-trained UNet and VAE, and only optimizes two deliberately-designed spatial feature adaptation (SFA) and temporal feature alignment (TFA) modules, in the decoder of UNet and VAE.
TRIP: Temporal Residual Learning with Image Noise Prior for Image-to-Video Diffusion Models
Next, TRIP executes a residual-like dual-path scheme for noise prediction: 1) a shortcut path that directly takes image noise prior as the reference noise of each frame to amplify the alignment between the first frame and subsequent frames; 2) a residual path that employs 3D-UNet over noised video and static image latent codes to enable inter-frame relational reasoning, thereby easing the learning of the residual noise for each frame.
VideoDrafter: Content-Consistent Multi-Scene Video Generation with LLM
The diffusion model incorporates the reference images as the condition and alignment to strengthen the content consistency of multi-scene videos.
CARIS: Context-Augmented Referring Image Segmentation
Technically, CARIS develops a context-aware mask decoder with sequential bidirectional cross-modal attention to integrate the linguistic features with visual context, which are then aligned with pixel-wise visual features.
Selective Volume Mixup for Video Action Recognition
In this paper, we propose a novel video augmentation strategy named Selective Volume Mixup (SV-Mix) to improve the generalization ability of deep models with limited training videos.
Learning Orthogonal Prototypes for Generalized Few-Shot Semantic Segmentation
POP builds a set of orthogonal prototypes, each of which represents a semantic class, and makes the prediction for each class separately based on the features projected onto its prototype.
Learning Neural Implicit Surfaces with Object-Aware Radiance Fields
Then, we build the geometric correspondence between 2D planes and 3D meshes by rasterization, and project the estimated object regions into 3D explicit object surfaces by aggregating the object information across multiple views.
PointClustering: Unsupervised Point Cloud Pre-Training Using Transformation Invariance in Clustering
Feature invariance under different data transformations, i. e., transformation invariance, can be regarded as a type of self-supervision for representation learning.
3D Human Pose Estimation With Spatio-Temporal Criss-Cross Attention
On this basis, we devise STCFormer by stacking multiple STC blocks and further integrate a new Structure-enhanced Positional Embedding (SPE) into STCFormer to take the structure of human body into consideration.
Ranked #6 on
3D Human Pose Estimation
on MPI-INF-3DHP
Dynamic Temporal Filtering in Video Models
The pre-determined kernel size severely limits the temporal receptive fields and the fixed weights treat each spatial location across frames equally, resulting in sub-optimal solution for long-range temporal modeling in natural scenes.
Explaining Cross-Domain Recognition with Interpretable Deep Classifier
In this paper, we ask the question: how much each sample in source domain contributes to the network's prediction on the samples from target domain.
SPE-Net: Boosting Point Cloud Analysis via Rotation Robustness Enhancement
In this paper, we propose a novel deep architecture tailored for 3D point cloud applications, named as SPE-Net.
Lightweight and Progressively-Scalable Networks for Semantic Segmentation
In this paper, we thoroughly analyze the design of convolutional blocks (the type of convolutions and the number of channels in convolutions), and the ways of interactions across multiple scales, all from lightweight standpoint for semantic segmentation.
Bi-Calibration Networks for Weakly-Supervised Video Representation Learning
The video-to-text/video-to-query projections over text prototypes/query vocabulary then start the text-to-query or query-to-text calibration to estimate the amendment to query or text.
Stand-Alone Inter-Frame Attention in Video Models
In this paper, we present a new recipe of inter-frame attention block, namely Stand-alone Inter-Frame Attention (SIFA), that novelly delves into the deformation across frames to estimate local self-attention on each spatial location.
Ranked #13 on
Action Recognition
on Something-Something V1
MLP-3D: A MLP-like 3D Architecture with Grouped Time Mixing
By deriving the novel grouped time mixing (GTM) operations, we equip the basic token-mixing MLP with the ability of temporal modeling.
Ranked #21 on
Action Recognition
on Something-Something V1
Silver-Bullet-3D at ManiSkill 2021: Learning-from-Demonstrations and Heuristic Rule-based Methods for Object Manipulation
This paper presents an overview and comparative analysis of our systems designed for the following two tracks in SAPIEN ManiSkill Challenge 2021: No Interaction Track: The No Interaction track targets for learning policies from pre-collected demonstration trajectories.
Condensing a Sequence to One Informative Frame for Video Recognition
This paper studies a two-step alternative that first condenses the video sequence to an informative "frame" and then exploits off-the-shelf image recognition system on the synthetic frame.
Motion-Focused Contrastive Learning of Video Representations
To this end, we compose a duet of exploiting the motion for data augmentation and feature learning in the regime of contrastive learning.
Representing Videos as Discriminative Sub-graphs for Action Recognition
For each action category, we execute online clustering to decompose the graph into sub-graphs on each scale through learning Gaussian Mixture Layer and select the discriminative sub-graphs as action prototypes for recognition.
Boosting Video Representation Learning with Multi-Faceted Integration
Video content is multifaceted, consisting of objects, scenes, interactions or actions.
Optimization Planning for 3D ConvNets
In this paper, we decompose the path into a series of training "states" and specify the hyper-parameters, e. g., learning rate and the length of input clips, in each state.
Learning to Localize Actions from Moments
In this paper, we introduce a new design of transfer learning type to learn action localization for a large set of action categories, but only on action moments from the categories of interest and temporal annotations of untrimmed videos from a small set of action classes.
SeCo: Exploring Sequence Supervision for Unsupervised Representation Learning
In this paper, we compose a trilogy of exploring the basic and generic supervision in the sequence from spatial, spatiotemporal and sequential perspectives.
Transferring and Regularizing Prediction for Semantic Segmentation
In the view of extremely expensive expert labeling, recent research has shown that the models trained on photo-realistic synthetic data (e. g., computer games) with computer-generated annotations can be adapted to real images.
Ranked #17 on
Domain Adaptation
on SYNTHIA-to-Cityscapes
Long Short-Term Relation Networks for Video Action Detection
It has been well recognized that modeling human-object or object-object relations would be helpful for detection task.
Scheduled Differentiable Architecture Search for Visual Recognition
Moreover, we enlarge the search space of SDAS particularly for video recognition by devising several unique operations to encode spatio-temporal dynamics and demonstrate the impact in affecting the architecture search of SDAS.
Gaussian Temporal Awareness Networks for Action Localization
Temporally localizing actions in a video is a fundamental challenge in video understanding.
Customizable Architecture Search for Semantic Segmentation
As a result, our CAS is able to search an optimized architecture with customized constraints.
vireoJD-MM at Activity Detection in Extended Videos
This notebook paper presents an overview and comparative analysis of our system designed for activity detection in extended videos (ActEV-PC) in ActivityNet Challenge 2019.
Trimmed Action Recognition, Dense-Captioning Events in Videos, and Spatio-temporal Action Localization with Focus on ActivityNet Challenge 2019
This notebook paper presents an overview and comparative analysis of our systems designed for the following three tasks in ActivityNet Challenge 2019: trimmed action recognition, dense-captioning events in videos, and spatio-temporal action localization.
Learning Spatio-Temporal Representation with Local and Global Diffusion
Diffusions effectively interact two aspects of information, i. e., localized and holistic, for more powerful way of representation learning.
Ranked #8 on
Action Recognition
on UCF101
Recurrent Tubelet Proposal and Recognition Networks for Action Detection
The RTP initializes action proposals of the start frame through a Region Proposal Network and then estimates the movements of proposals in next frame in a recurrent manner.
Fully Convolutional Adaptation Networks for Semantic Segmentation
The recent advances in deep neural networks have convincingly demonstrated high capability in learning vision models on large datasets.
To Create What You Tell: Generating Videos from Captions
In this paper, we present a novel Temporal GANs conditioning on Captions, namely TGANs-C, in which the input to the generator network is a concatenation of a latent noise vector and caption embedding, and then is transformed into a frame sequence with 3D spatio-temporal convolutions.
Deep Semantic Hashing with Generative Adversarial Networks
Specifically, a novel deep semantic hashing with GANs (DSH-GANs) is presented, which mainly consists of four components: a deep convolution neural networks (CNN) for learning image representations, an adversary stream to distinguish synthetic images from real ones, a hash stream for encoding image representations to hash codes and a classification stream.
Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks
In this paper, we devise multiple variants of bottleneck building blocks in a residual learning framework by simulating $3\times3\times3$ convolutions with $1\times3\times3$ convolutional filters on spatial domain (equivalent to 2D CNN) plus $3\times1\times1$ convolutions to construct temporal connections on adjacent feature maps in time.
Ranked #7 on
Action Recognition
on Sports-1M
Deep Quantization: Encoding Convolutional Activations with Deep Generative Model
In this paper, we present Fisher Vector encoding with Variational Auto-Encoder (FV-VAE), a novel deep architecture that quantizes the local activations of convolutional layer in a deep generative model, by training them in an end-to-end manner.
Boosting Image Captioning with Attributes
Automatically describing an image with a natural language has been an emerging challenge in both fields of computer vision and natural language processing.
