Search Results for author: Yunbo Wang

Found 35 papers, 22 papers with code

Vid2Act: Activate Offline Videos for Visual RL

no code implementations6 Jun 2023 Minting Pan, Yitao Zheng, Wendong Zhang, Yunbo Wang, Xiaokang Yang

Pretraining RL models on offline video datasets is a promising way to improve their training efficiency in online tasks, but challenging due to the inherent mismatch in tasks, dynamics, and behaviors across domains.

Knowledge Distillation

Collaborative World Models: An Online-Offline Transfer RL Approach

no code implementations24 May 2023 Qi Wang, Junming Yang, Yunbo Wang, Xin Jin, Wenjun Zeng, Xiaokang Yang

Training visual reinforcement learning (RL) models in offline datasets is challenging due to overfitting issues in representation learning and overestimation problems in value function.

Reinforcement Learning (RL) Representation Learning +1

Unsupervised Object-Centric Voxelization for Dynamic Scene Understanding

no code implementations30 Apr 2023 Siyu Gao, Yanpeng Zhao, Yunbo Wang, Xiaokang Yang

Understanding the compositional dynamics of multiple objects in unsupervised visual environments is challenging, and existing object-centric representation learning methods often ignore 3D consistency in scene decomposition.

Neural Rendering Novel View Synthesis +2

Model-Based Reinforcement Learning with Isolated Imaginations

1 code implementation27 Mar 2023 Minting Pan, Xiangming Zhu, Yunbo Wang, Xiaokang Yang

On top of our previous work, we further consider the sparse dependencies between controllable and noncontrollable states, address the training collapse problem of state decoupling, and validate our approach in transfer learning setups.

Autonomous Driving Model-based Reinforcement Learning +3

Predictive Experience Replay for Continual Visual Control and Forecasting

1 code implementation12 Mar 2023 Wendong Zhang, Geng Chen, Xiangming Zhu, Siyu Gao, Yunbo Wang, Xiaokang Yang

In this paper, we present a new continual learning approach for visual dynamics modeling and explore its efficacy in visual control and forecasting.

Continual Learning Model-based Reinforcement Learning +2

Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Models

2 code implementations27 May 2022 Minting Pan, Xiangming Zhu, Yunbo Wang, Xiaokang Yang

First, by optimizing the inverse dynamics, we encourage the world model to learn controllable and noncontrollable sources of spatiotemporal changes on isolated state transition branches.

Autonomous Driving Decision Making

MetaSets: Meta-Learning on Point Sets for Generalizable Representations

no code implementations CVPR 2021 Chao Huang, Zhangjie Cao, Yunbo Wang, Jianmin Wang, Mingsheng Long

It is a challenging problem due to the substantial geometry shift from simulated to real data, such that most existing 3D models underperform due to overfitting the complete geometries in the source domain.

Domain Generalization Meta-Learning

Continual Predictive Learning from Videos

1 code implementation CVPR 2022 Geng Chen, Wendong Zhang, Han Lu, Siyu Gao, Yunbo Wang, Mingsheng Long, Xiaokang Yang

Can we develop predictive learning algorithms that can deal with more realistic, non-stationary physical environments?

Continual Learning Video Prediction

NeuroFluid: Fluid Dynamics Grounding with Particle-Driven Neural Radiance Fields

no code implementations3 Mar 2022 Shanyan Guan, Huayu Deng, Yunbo Wang, Xiaokang Yang

Deep learning has shown great potential for modeling the physical dynamics of complex particle systems such as fluids.

Out-of-Domain Human Mesh Reconstruction via Dynamic Bilevel Online Adaptation

1 code implementation7 Nov 2021 Shanyan Guan, Jingwei Xu, Michelle Z. He, Yunbo Wang, Bingbing Ni, Xiaokang Yang

We consider a new problem of adapting a human mesh reconstruction model to out-of-domain streaming videos, where performance of existing SMPL-based models are significantly affected by the distribution shift represented by different camera parameters, bone lengths, backgrounds, and occlusions.

3D Absolute Human Pose Estimation Bilevel Optimization

ModeRNN: Harnessing Spatiotemporal Mode Collapse in Unsupervised Predictive Learning

1 code implementation8 Oct 2021 Zhiyu Yao, Yunbo Wang, Haixu Wu, Jianmin Wang, Mingsheng Long

To this end, we propose ModeRNN, which introduces a novel method to learn structured hidden representations between recurrent states.

Inductive Bias

Context-Aware Image Inpainting with Learned Semantic Priors

1 code implementation14 Jun 2021 Wendong Zhang, Junwei Zhu, Ying Tai, Yunbo Wang, Wenqing Chu, Bingbing Ni, Chengjie Wang, Xiaokang Yang

Based on the semantic priors, we further propose a context-aware image inpainting model, which adaptively integrates global semantics and local features in a unified image generator.

Image Inpainting Knowledge Distillation

MetaSets:Meta-Learning on Point Sets for Generalizable Representations

no code implementations CVPR 2021 Chao Huang, Zhangjie Cao, Yunbo Wang, Jianmin Wang, Mingsheng Long

It is a challenging problem due to the substantial geometry shift from simulated to real data, such that most existing 3D models underperform due to overfitting the complete geometries in the source domain.

Domain Generalization

Bilevel Online Adaptation for Out-of-Domain Human Mesh Reconstruction

1 code implementation CVPR 2021 Shanyan Guan, Jingwei Xu, Yunbo Wang, Bingbing Ni, Xiaokang Yang

This paper considers a new problem of adapting a pre-trained model of human mesh reconstruction to out-of-domain streaming videos.

3D Human Pose Estimation

PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning

3 code implementations17 Mar 2021 Yunbo Wang, Haixu Wu, Jianjin Zhang, Zhifeng Gao, Jianmin Wang, Philip S. Yu, Mingsheng Long

This paper models these structures by presenting PredRNN, a new recurrent network, in which a pair of memory cells are explicitly decoupled, operate in nearly independent transition manners, and finally form unified representations of the complex environment.

 Ranked #1 on Video Prediction on KTH (Cond metric)

Video Prediction Weather Forecasting

Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer

2 code implementations14 Dec 2020 Jian Liang, Dapeng Hu, Yunbo Wang, Ran He, Jiashi Feng

Furthermore, we propose a new labeling transfer strategy, which separates the target data into two splits based on the confidence of predictions (labeling information), and then employ semi-supervised learning to improve the accuracy of less-confident predictions in the target domain.

Classification General Classification +3

Towards Good Practices of U-Net for Traffic Forecasting

1 code implementation4 Dec 2020 Jingwei Xu, Jianjin Zhang, Zhiyu Yao, Yunbo Wang

This technical report presents a solution for the 2020 Traffic4Cast Challenge.

Unsupervised Transfer Learning for Spatiotemporal Predictive Networks

1 code implementation ICML 2020 Zhiyu Yao, Yunbo Wang, Mingsheng Long, Jian-Min Wang

This paper explores a new research problem of unsupervised transfer learning across multiple spatiotemporal prediction tasks.

Transfer Learning

Probabilistic Video Prediction From Noisy Data With a Posterior Confidence

no code implementations CVPR 2020 Yunbo Wang, Jiajun Wu, Mingsheng Long, Joshua B. Tenenbaum

It is also challenging because it involves two levels of uncertainty: the perceptual uncertainty from noisy observations and the dynamics uncertainty in forward modeling.

Video Prediction

VideoDG: Generalizing Temporal Relations in Videos to Novel Domains

1 code implementation8 Dec 2019 Zhiyu Yao, Yunbo Wang, Jianmin Wang, Philip S. Yu, Mingsheng Long

This paper introduces video domain generalization where most video classification networks degenerate due to the lack of exposure to the target domains of divergent distributions.

Action Recognition Data Augmentation +4

DualSMC: Tunneling Differentiable Filtering and Planning under Continuous POMDPs

1 code implementation28 Sep 2019 Yunbo Wang, Bo Liu, Jiajun Wu, Yuke Zhu, Simon S. Du, Li Fei-Fei, Joshua B. Tenenbaum

A major difficulty of solving continuous POMDPs is to infer the multi-modal distribution of the unobserved true states and to make the planning algorithm dependent on the perceived uncertainty.

Continuous Control

Z-Order Recurrent Neural Networks for Video Prediction

no code implementations IEEE International Conference on Multimedia and Expo (ICME) 2019 Jianjin Zhang, Yunbo Wang, Mingsheng Long, Wang Jianmin, Philip S Yu

First, we propose a new RNN architecture for modeling the deterministic dynamics, which updates hidden states along a z-order curve to enhance the consistency of the features of mirrored layers.

 Ranked #1 on Video Prediction on KTH (Cond metric)

Video Prediction

Eidetic 3D LSTM: A Model for Video Prediction and Beyond

3 code implementations ICLR 2019 Yunbo Wang, Lu Jiang, Ming-Hsuan Yang, Li-Jia Li, Mingsheng Long, Li Fei-Fei

We first evaluate the E3D-LSTM network on widely-used future video prediction datasets and achieve the state-of-the-art performance.

 Ranked #1 on Video Prediction on KTH (Cond metric)

Activity Recognition Video Prediction +1

Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spatiotemporal Dynamics

3 code implementations CVPR 2019 Yunbo Wang, Jianjin Zhang, Hongyu Zhu, Mingsheng Long, Jian-Min Wang, Philip S. Yu

Natural spatiotemporal processes can be highly non-stationary in many ways, e. g. the low-level non-stationarity such as spatial correlations or temporal dependencies of local pixel values; and the high-level variations such as the accumulation, deformation or dissipation of radar echoes in precipitation forecasting.

Time Series Forecasting Video Prediction

Recognizing Partial Biometric Patterns

1 code implementation17 Oct 2018 Lingxiao He, Zhenan Sun, Yuhao Zhu, Yunbo Wang

Biometric recognition on partial captured targets is challenging, where only several partial observations of objects are available for matching.

Dictionary Learning Face Recognition +1

PredRNN: Recurrent Neural Networks for Predictive Learning using Spatiotemporal LSTMs

no code implementations NeurIPS 2017 Yunbo Wang, Mingsheng Long, Jian-Min Wang, Zhifeng Gao, Philip S. Yu

The core of this network is a new Spatiotemporal LSTM (ST-LSTM) unit that extracts and memorizes spatial and temporal representations simultaneously.

Video Prediction

Cannot find the paper you are looking for? You can Submit a new open access paper.