PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization

We introduce Pixel-aligned Implicit Function (PIFu), a highly effective implicit representation that locally aligns pixels of 2D images with the global context of their corresponding 3D object. Using PIFu, we propose an end-to-end deep learning method for digitizing highly detailed clothed humans that can infer both 3D surface and texture from a single image, and optionally, multiple input images. Highly intricate shapes, such as hairstyles, clothing, as well as their variations and deformations can be digitized in a unified way. Compared to existing representations used for 3D deep learning, PIFu can produce high-resolution surfaces including largely unseen regions such as the back of a person. In particular, it is memory efficient unlike the voxel representation, can handle arbitrary topology, and the resulting surface is spatially aligned with the input image. Furthermore, while previous techniques are designed to process either a single image or multiple views, PIFu extends naturally to arbitrary number of views. We demonstrate high-resolution and robust reconstructions on real world images from the DeepFashion dataset, which contains a variety of challenging clothing types. Our method achieves state-of-the-art performance on a public benchmark and outperforms the prior work for clothed human digitization from a single image.

PDF Abstract ICCV 2019 PDF ICCV 2019 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
3D Object Reconstruction From A Single Image BUFF PIFu Point-to-surface distance (cm) 1.15 # 4
Chamfer (cm) 1.14 # 3
Surface normal consistency 0.0928 # 3
3D Human Reconstruction CAPE PIFu (THuman2.0) Chamfer (cm) 3.573 # 4
P2S (cm) 1.483 # 2
NC 0.186 # 4
3D Object Reconstruction From A Single Image RenderPeople PIFu Point-to-surface distance (cm) 1.52 # 2
Chamfer (cm) 1.5 # 2
Surface normal consistency 0.084 # 2
3D Object Reconstruction RenderPeople PIFu (3 views) Surface normal consistency 0.094 # 1
Point-to-surface distance (cm) 0.554 # 1
Chamfer (cm) 0.567 # 1

Methods


No methods listed for this paper. Add relevant methods here