Search Results for author:
Found 12 papers, 8 papers with code
TRIP: Temporal Residual Learning with Image Noise Prior for Image-to-Video Diffusion Models
Next, TRIP executes a residual-like dual-path scheme for noise prediction: 1) a shortcut path that directly takes image noise prior as the reference noise of each frame to amplify the alignment between the first frame and subsequent frames; 2) a residual path that employs 3D-UNet over noised video and static image latent codes to enable inter-frame relational reasoning, thereby easing the learning of the residual noise for each frame.
Learning Spatial Adaptation and Temporal Coherence in Diffusion Models for Video Super-Resolution
Technically, SATeCo freezes all the parameters of the pre-trained UNet and VAE, and only optimizes two deliberately-designed spatial feature adaptation (SFA) and temporal feature alignment (TFA) modules, in the decoder of UNet and VAE.
VideoDrafter: Content-Consistent Multi-Scene Video Generation with LLM
The diffusion model incorporates the reference images as the condition and alignment to strengthen the content consistency of multi-scene videos.
PointClustering: Unsupervised Point Cloud Pre-Training Using Transformation Invariance in Clustering
Feature invariance under different data transformations, i. e., transformation invariance, can be regarded as a type of self-supervision for representation learning.
Dynamic Temporal Filtering in Video Models
The pre-determined kernel size severely limits the temporal receptive fields and the fixed weights treat each spatial location across frames equally, resulting in sub-optimal solution for long-range temporal modeling in natural scenes.
Bi-Calibration Networks for Weakly-Supervised Video Representation Learning
The video-to-text/video-to-query projections over text prototypes/query vocabulary then start the text-to-query or query-to-text calibration to estimate the amendment to query or text.
Stand-Alone Inter-Frame Attention in Video Models
In this paper, we present a new recipe of inter-frame attention block, namely Stand-alone Inter-Frame Attention (SIFA), that novelly delves into the deformation across frames to estimate local self-attention on each spatial location.
Ranked #13 on
Action Recognition
on Something-Something V1
Silver-Bullet-3D at ManiSkill 2021: Learning-from-Demonstrations and Heuristic Rule-based Methods for Object Manipulation
This paper presents an overview and comparative analysis of our systems designed for the following two tracks in SAPIEN ManiSkill Challenge 2021: No Interaction Track: The No Interaction track targets for learning policies from pre-collected demonstration trajectories.
Learning to Localize Actions from Moments
In this paper, we introduce a new design of transfer learning type to learn action localization for a large set of action categories, but only on action moments from the categories of interest and temporal annotations of untrimmed videos from a small set of action classes.
Gaussian Temporal Awareness Networks for Action Localization
Temporally localizing actions in a video is a fundamental challenge in video understanding.
vireoJD-MM at Activity Detection in Extended Videos
This notebook paper presents an overview and comparative analysis of our system designed for activity detection in extended videos (ActEV-PC) in ActivityNet Challenge 2019.
