Search Results for author:
Found 35 papers, 22 papers with code
Vid2Act: Activate Offline Videos for Visual RL
Pretraining RL models on offline video datasets is a promising way to improve their training efficiency in online tasks, but challenging due to the inherent mismatch in tasks, dynamics, and behaviors across domains.
Collaborative World Models: An Online-Offline Transfer RL Approach
Training visual reinforcement learning (RL) models in offline datasets is challenging due to overfitting issues in representation learning and overestimation problems in value function.
Unsupervised Object-Centric Voxelization for Dynamic Scene Understanding
Understanding the compositional dynamics of multiple objects in unsupervised visual environments is challenging, and existing object-centric representation learning methods often ignore 3D consistency in scene decomposition.
Model-Based Reinforcement Learning with Isolated Imaginations
On top of our previous work, we further consider the sparse dependencies between controllable and noncontrollable states, address the training collapse problem of state decoupling, and validate our approach in transfer learning setups.
Predictive Experience Replay for Continual Visual Control and Forecasting
In this paper, we present a new continual learning approach for visual dynamics modeling and explore its efficacy in visual control and forecasting.
Improving Masked Autoencoders by Learning Where to Mask
Masked image modeling is a promising self-supervised learning method for visual data.
Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Models
First, by optimizing the inverse dynamics, we encourage the world model to learn controllable and noncontrollable sources of spatiotemporal changes on isolated state transition branches.
MetaSets: Meta-Learning on Point Sets for Generalizable Representations
It is a challenging problem due to the substantial geometry shift from simulated to real data, such that most existing 3D models underperform due to overfitting the complete geometries in the source domain.
Continual Predictive Learning from Videos
Can we develop predictive learning algorithms that can deal with more realistic, non-stationary physical environments?
NeuroFluid: Fluid Dynamics Grounding with Particle-Driven Neural Radiance Fields
Deep learning has shown great potential for modeling the physical dynamics of complex particle systems such as fluids.
Fully Context-Aware Image Inpainting with a Learned Semantic Pyramid
We train the prior learner and the image generator as a unified model without any post-processing.
Learning Transferable Features for Point Cloud Detection via 3D Contrastive Co-training
First, 3D-CoCo is inspired by our observation that the bird-eye-view (BEV) features are more transferable than low-level geometry features.
Out-of-Domain Human Mesh Reconstruction via Dynamic Bilevel Online Adaptation
We consider a new problem of adapting a human mesh reconstruction model to out-of-domain streaming videos, where performance of existing SMPL-based models are significantly affected by the distribution shift represented by different camera parameters, bone lengths, backgrounds, and occlusions.
Ranked #1 on
3D Absolute Human Pose Estimation
on Surreal
ModeRNN: Harnessing Spatiotemporal Mode Collapse in Unsupervised Predictive Learning
To this end, we propose ModeRNN, which introduces a novel method to learn structured hidden representations between recurrent states.
Context-Aware Image Inpainting with Learned Semantic Priors
Based on the semantic priors, we further propose a context-aware image inpainting model, which adaptively integrates global semantics and local features in a unified image generator.
MetaSets：Meta-Learning on Point Sets for Generalizable Representations
It is a challenging problem due to the substantial geometry shift from simulated to real data, such that most existing 3D models underperform due to overfitting the complete geometries in the source domain.
Bilevel Online Adaptation for Out-of-Domain Human Mesh Reconstruction
This paper considers a new problem of adapting a pre-trained model of human mesh reconstruction to out-of-domain streaming videos.
Ranked #29 on
3D Human Pose Estimation
on 3DPW
PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning
This paper models these structures by presenting PredRNN, a new recurrent network, in which a pair of memory cells are explicitly decoupled, operate in nearly independent transition manners, and finally form unified representations of the complex environment.
Ranked #1 on
Video Prediction
on KTH
(Cond metric)
Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer
Furthermore, we propose a new labeling transfer strategy, which separates the target data into two splits based on the confidence of predictions (labeling information), and then employ semi-supervised learning to improve the accuracy of less-confident predictions in the target domain.
Towards Good Practices of U-Net for Traffic Forecasting
This technical report presents a solution for the 2020 Traffic4Cast Challenge.
Unsupervised Transfer Learning for Spatiotemporal Predictive Networks
This paper explores a new research problem of unsupervised transfer learning across multiple spatiotemporal prediction tasks.
Probabilistic Video Prediction From Noisy Data With a Posterior Confidence
It is also challenging because it involves two levels of uncertainty: the perceptual uncertainty from noisy observations and the dynamics uncertainty in forward modeling.
A Balanced and Uncertainty-aware Approach for Partial Domain Adaptation
On one hand, negative transfer results in misclassification of target samples to the classes only present in the source domain.
Ranked #2 on
Partial Domain Adaptation
on DomainNet
VideoDG: Generalizing Temporal Relations in Videos to Novel Domains
This paper introduces video domain generalization where most video classification networks degenerate due to the lack of exposure to the target domains of divergent distributions.
DualSMC: Tunneling Differentiable Filtering and Planning under Continuous POMDPs
A major difficulty of solving continuous POMDPs is to infer the multi-modal distribution of the unobserved true states and to make the planning algorithm dependent on the perceived uncertainty.
DELTA: A DEep learning based Language Technology plAtform
In this paper we present DELTA, a deep learning based language technology platform.
Ranked #3 on
Text Classification
on Yahoo! Answers
Z-Order Recurrent Neural Networks for Video Prediction
First, we propose a new RNN architecture for modeling the deterministic dynamics, which updates hidden states along a z-order curve to enhance the consistency of the features of mirrored layers.
Ranked #1 on
Video Prediction
on KTH
(Cond metric)
Eidetic 3D LSTM: A Model for Video Prediction and Beyond
We first evaluate the E3D-LSTM network on widely-used future video prediction datasets and achieve the state-of-the-art performance.
Ranked #1 on
Video Prediction
on KTH
(Cond metric)
Spatiotemporal Pyramid Network for Video Action Recognition
From the technical perspective, we introduce the spatiotemporal compact bilinear operator into video analysis tasks.
Multi-Task Learning of Generalizable Representations for Video Action Recognition
Rev2Net is shown to be effective on the classic action recognition task.
Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spatiotemporal Dynamics
Natural spatiotemporal processes can be highly non-stationary in many ways, e. g. the low-level non-stationarity such as spatial correlations or temporal dependencies of local pixel values; and the high-level variations such as the accumulation, deformation or dissipation of radar echoes in precipitation forecasting.
Ranked #5 on
Video Prediction
on Human3.6M
Recognizing Partial Biometric Patterns
Biometric recognition on partial captured targets is challenging, where only several partial observations of objects are available for matching.
Predcnn: Predictive learning with cascade convolutions
Predicting future frames in videos remains an unsolved but challenging problem.
Ranked #3 on
Pose Prediction
on Filtered NTU RGB+D
PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning
We present PredRNN++, an improved recurrent network for video predictive learning.
Ranked #1 on
Video Prediction
on KTH
(Cond metric)
PredRNN: Recurrent Neural Networks for Predictive Learning using Spatiotemporal LSTMs
The core of this network is a new Spatiotemporal LSTM (ST-LSTM) unit that extracts and memorizes spatial and temporal representations simultaneously.
Ranked #6 on
Video Prediction
on Human3.6M
