1 code implementation • 9 Dec 2024 • Ruihan Gao, Kangle Deng, Gengshan Yang, Wenzhen Yuan, Jun-Yan Zhu
We design a lightweight 3D texture field to synthesize visual and tactile textures, guided by 2D diffusion model priors on both visual and tactile domains.
no code implementations • 21 Oct 2024 • Gengshan Yang, Andrea Bajcsy, Shunsuke Saito, Angjoo Kanazawa
We present Agent-to-Sim (ATS), a framework for learning interactive behavior models of 3D agents from casual longitudinal video collections.
no code implementations • 30 Sep 2024 • Jeff Tan, Donglai Xiang, Shubham Tulsiani, Deva Ramanan, Gengshan Yang
We present a method to reconstruct time-consistent human body models from monocular videos, focusing on extremely loose clothing or handheld object interactions.
no code implementations • CVPR 2024 • Nikhil Keetha, Jay Karhade, Krishna Murthy Jatavallabhula, Gengshan Yang, Sebastian Scherer, Deva Ramanan, Jonathon Luiten
Dense simultaneous localization and mapping (SLAM) is crucial for robotics and augmented reality applications.
2 code implementations • 4 Dec 2023 • Nikhil Keetha, Jay Karhade, Krishna Murthy Jatavallabhula, Gengshan Yang, Sebastian Scherer, Deva Ramanan, Jonathon Luiten
Dense simultaneous localization and mapping (SLAM) is crucial for robotics and augmented reality applications.
2 code implementations • CVPR 2023 • Gengshan Yang, Chaoyang Wang, N Dinesh Reddy, Deva Ramanan
Building animatable 3D models is challenging due to the need for 3D scans, laborious registration, and manual rigging, which are difficult to scale to arbitrary categories.
3D Shape Reconstruction from Videos
Dynamic Reconstruction
+1
1 code implementation • ICCV 2023 • Chonghyuk Song, Gengshan Yang, Kangle Deng, Jun-Yan Zhu, Deva Ramanan
Given a minute-long RGBD video of people interacting with their pets, we render the scene from novel camera trajectories derived from the in-scene motion of actors: (1) egocentric cameras that simulate the point of view of a target actor and (2) 3rd-person cameras that follow the actor.
2 code implementations • CVPR 2023 • Kangle Deng, Gengshan Yang, Deva Ramanan, Jun-Yan Zhu
We propose pix2pix3D, a 3D-aware conditional generative model for controllable photorealistic image synthesis.
no code implementations • CVPR 2023 • Jeff Tan, Gengshan Yang, Deva Ramanan
We present a method for reconstructing articulated 3D models from videos in real-time, without test-time optimization or manual 3D supervision at training time.
1 code implementation • ICCV 2023 • Gengshan Yang, Shuo Yang, John Z. Zhang, Zachary Manchester, Deva Ramanan
Given monocular videos, we build 3D models of articulated objects and environments whose 3D configurations satisfy dynamics and contact constraints.
1 code implementation • CVPR 2022 • Gengshan Yang, Minh Vo, Natalia Neverova, Deva Ramanan, Andrea Vedaldi, Hanbyul Joo
Our key insight is to merge three schools of thought; (1) classic deformable shape models that make use of articulated bones and blend skinning, (2) volumetric neural radiance fields (NeRFs) that are amenable to gradient-based optimization, and (3) canonical embeddings that generate correspondences between pixels and an articulated model.
1 code implementation • NeurIPS 2021 • Gengshan Yang, Deqing Sun, Varun Jampani, Daniel Vlasic, Forrester Cole, Ce Liu, Deva Ramanan
The surface embeddings are implemented as coordinate-based MLPs that are fit to each video via consistency and contrastive reconstruction losses. Experimental results show that ViSER compares favorably against prior work on challenging videos of humans with loose clothing and unusual poses as well as animals videos from DAVIS and YTVOS.
1 code implementation • NeurIPS 2021 • Jason Y. Zhang, Gengshan Yang, Shubham Tulsiani, Deva Ramanan
NeRS learns a neural shape representation of a closed surface that is diffeomorphic to a sphere, guaranteeing water-tight reconstructions.
1 code implementation • CVPR 2021 • Gengshan Yang, Deqing Sun, Varun Jampani, Daniel Vlasic, Forrester Cole, Huiwen Chang, Deva Ramanan, William T. Freeman, Ce Liu
Remarkable progress has been made in 3D reconstruction of rigid structures from a video or a collection of images.
1 code implementation • CVPR 2021 • Gengshan Yang, Deva Ramanan
Geometric motion segmentation algorithms, however, generalize to novel scenes, but have yet to achieve comparable performance to appearance-based ones, due to noisy motion estimations and degenerate motion configurations.
1 code implementation • CVPR 2020 • Gengshan Yang, Deva Ramanan
We describe an approach for upgrading 2D optical flow to 3D scene flow.
2 code implementations • CVPR 2019 • Gengshan Yang, Joshua Manela, Michael Happold, Deva Ramanan
We explore the problem of real-time stereo matching on high-res imagery.
1 code implementation • 12 Dec 2019 • Gengshan Yang, Peiyun Hu, Deva Ramanan
Such approaches cannot diagnose when failures might occur.
2 code implementations • NeurIPS 2019 • Gengshan Yang, Deva Ramanan
As a result, SOTA networks also employ various heuristics designed to limit volumetric processing, leading to limited accuracy and overfitting.
Ranked #14 on
Optical Flow Estimation
on KITTI 2015 (train)