Volumetric Capture of Humans with a Single RGBD Camera via Semi-Parametric Learning

no code implementations • CVPR 2019 • Rohit Pandey, Anastasia Tkach, Shuoran Yang, Pavel Pidlypenskyi, Jonathan Taylor, Ricardo Martin-Brualla, Andrea Tagliasacchi, George Papandreou, Philip Davidson, Cem Keskin, Shahram Izadi, Sean Fanello

The key insight is to leverage previously seen "calibration" images of a given user to extrapolate what should be rendered in a novel viewpoint from the data available in the sensor.

LookinGood: Enhancing Performance Capture with Real-time Neural Re-Rendering

no code implementations • 12 Nov 2018 • Ricardo Martin-Brualla, Rohit Pandey, Shuoran Yang, Pavel Pidlypenskyi, Jonathan Taylor, Julien Valentin, Sameh Khamis, Philip Davidson, Anastasia Tkach, Peter Lincoln, Adarsh Kowdle, Christoph Rhemann, Dan B. Goldman, Cem Keskin, Steve Seitz, Shahram Izadi, Sean Fanello

We take the novel approach to augment such real-time performance capture systems with a deep architecture that takes a rendering from an arbitrary viewpoint, and jointly performs completion, super resolution, and denoising of the imagery in real-time.

Denoising Super-Resolution

Efficient 6-DoF Tracking of Handheld Objects from an Egocentric Viewpoint

no code implementations • ECCV 2018 • Rohit Pandey, Pavel Pidlypenskyi, Shuoran Yang, Christine Kaeser-Chen

Virtual and augmented reality technologies have seen significant growth in the past few years.

Egocentric 6-DoF Tracking of Small Handheld Objects

no code implementations • 16 Apr 2018 • Rohit Pandey, Pavel Pidlypenskyi, Shuoran Yang, Christine Kaeser-Chen

Virtual and augmented reality technologies have seen significant growth in the past few years.

Real-time Egocentric Gesture Recognition on Mobile Head Mounted Displays

no code implementations • 13 Dec 2017 • Rohit Pandey, Marie White, Pavel Pidlypenskyi, Xue Wang, Christine Kaeser-Chen

Mobile virtual reality (VR) head mounted displays (HMD) have become popular among consumers in recent years.

Gesture Recognition Mixed Reality