Search Results for author:
Found 28 papers, 10 papers with code
Real-Time High-Resolution Background Matting
We introduce a real-time, high-resolution background replacement technique which operates at 30fps in 4K resolution, and 60fps for HD on a modern GPU.
Background Matting: The World is Your Green Screen
To bridge the domain gap to real imagery with no labeling, we train another matting network guided by the first network and by a discriminator that judges the quality of composites.
Ranked #1 on
Image Matting
on Adobe Matting
HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video
Our method optimizes for a volumetric representation of the person in a canonical T-pose, in concert with a motion field that maps the estimated canonical representation to every frame of the video via backward warps.
Lifespan Age Transformation Synthesis
Most existing aging methods are limited to changing the texture, overlooking transformations in head shape that occur during the human aging and growth process.
StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation
We introduce a high resolution, 3D-consistent image and shape generation technique which we call StyleSDF.
The Cone of Silence: Speech Separation by Localization
Given a multi-microphone recording of an unknown number of speakers talking concurrently, we simultaneously localize the sources and separate the individual speakers.
Audio to Body Dynamics
We present a method that gets as input an audio of violin or piano playing, and outputs a video of skeleton predictions which are further used to animate an avatar.
TryOnDiffusion: A Tale of Two UNets
Given two images depicting a person and a garment worn by another person, our goal is to generate a visualization of how the garment might look on the input person.
Reconstructing NBA Players
Based on these models, we introduce a new method that takes as input a single photo of a clothed player in any basketball pose and outputs a high resolution mesh and 3D pose for that player.
TryOnGAN: Body-Aware Try-On via Layered Interpolation
Previous methods mostly focused on texture transfer via paired data training, while overlooking body shape deformations, skin color, and seamless blending of garment with the person.
Soccer on Your Tabletop
We present a system that transforms a monocular video of a soccer game into a moving 3D reconstruction, in which the players and field can be rendered interactively with a 3D viewer or through an Augmented Reality device.
Level Playing Field for Million Scale Face Recognition
Some key discoveries: 1) algorithms, trained on MF2, were able to achieve state of the art and comparable results to algorithms trained on massive private sets, 2) some outperformed themselves once trained on MF2, 3) invariance to aging suffers from low accuracies as in MegaFace, identifying the need for larger age variations possibly within identities or adjustment of algorithms in future testings.
The MegaFace Benchmark: 1 Million Faces for Recognition at Scale
Our key observations are that testing at the million scale reveals big performance differences (of algorithms that perform similarly well on smaller scale) and that age invariant recognition as well as pose are still challenging for most.
What Makes Kevin Spacey Look Like Kevin Spacey
We reconstruct a controllable model of a person from a large photo collection that captures his or her {\em persona}, i. e., physical appearance and behavior.
Video to Fully Automatic 3D Hair Model
In this paper, we describe a system that can completely automatically create a reconstruction from any video (even a selfie video), and we don't require specific views, since taking your -90 degree, 90 degree, and full back views is not feasible in a selfie capture.
3D Face Hallucination from a Single Depth Frame
We further combine the input depth frame with the matched database shapes into a single mesh that results in a high-resolution shape of the input person.
Head Reconstruction from Internet Photos
Our method is to gradually "grow" the head mesh starting from the frontal face and extending to the rest of views using photometric stereo constraints.
Photo Wake-Up: 3D Character Animation from a Single Photo
The key contributions of this paper are: 1) an application of viewing and animating humans in single photos in 3D, 2) a novel 2D warping method to deform a posable template body model to fit the person's complex silhouette to create an animatable mesh, and 3) a method for handling partial self occlusions.
Illumination-Aware Age Progression
We present an approach that takes a single photograph of a child as input and automatically produces a series of age-progressed outputs between 1 and 80 years of age, accounting for pose, expression, and illumination.
What Makes Tom Hanks Look Like Tom Hanks
We reconstruct a controllable model of a person from a large photo collection that captures his or her persona, i. e., physical appearance and behavior.
Vid2Actor: Free-viewpoint Animatable Person Synthesis from Video in the Wild
At the core of our method is a volumetric 3D human representation reconstructed with a deep network trained on input video, enabling novel pose/view synthesis.
A Light Stage on Every Desk
Whereas existing light stages require expensive, room-scale spherical capture gantries and exist in only a few labs in the world, we demonstrate how to acquire useful data from a normal TV or desktop monitor.
PersonNeRF: Personalized Reconstruction from Photo Collections
We present PersonNeRF, a method that takes a collection of photos of a subject (e. g. Roger Federer) captured across multiple years with arbitrary body poses and appearances, and enables rendering the subject with arbitrary novel combinations of viewpoint, body pose, and appearance.
DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion
We fine-tune on a collection of fashion videos from the UBC Fashion dataset.
Total Selfie: Generating Full-Body Selfies
We present a method to generate full-body selfies from photographs originally taken at arms length.
DreamPose: Fashion Video Synthesis with Stable Diffusion
We fine-tune on a collection of fashion videos from the UBC Fashion dataset.
Animating Street View
We present a system that automatically brings street view imagery to life by populating it with naturally behaving, animated pedestrians and vehicles.
HRTF Estimation in the Wild
Our approach offers an efficient and accessible method for deriving personalized HRTFs and has the potential to greatly improve spatial audio experiences.
