Towards this end, both explicit and implicit 3D representations are heavily studied for a holistic modeling and capture of the whole human (e. g., body, clothing, face and hair), but neither representation is an optimal choice in terms of representation efficacy since different parts of the human avatar have different modeling desiderata.
We present SCULPT, a novel 3D generative model for clothed and textured 3D meshes of humans.
While the best recent methods generate 3D animations that are synchronized with the input audio, they largely ignore the impact of emotions on facial expressions.
We use raw MVS scans as supervision during training, but, once trained, TEMPEH directly predicts 3D heads in dense correspondence without requiring scans.
Using novel 4D scans of feet, we train a model with an extended kinematic tree that captures the range of motion of the toes.
Training of BMnet is performed on data from real human subjects, and augmented with a novel adversarial body simulator (ABS) that finds and synthesizes challenging body shapes.
Building on this insight, we propose SCARF (Segmented Clothed Avatar Radiance Field), a hybrid model combining a mesh-based body with a neural radiance field.
Our experimental results show significant improvement compared to state-of-the-art methods on albedo estimation, both in terms of accuracy and fairness.
While EMOCA achieves 3D reconstruction errors that are on par with the current best methods, it significantly outperforms them in terms of the quality of the reconstructed expression and the perceived emotional content.
Ranked #12 on 3D Face Reconstruction on REALY (side-view)
To this end, we take advantage of a face recognition network pretrained on a large-scale 2D image dataset, which provides distinct features for different faces and is robust to expression, illumination, and camera changes.
Ranked #2 on 3D Face Reconstruction on NoW Benchmark
Synthesizing images of a person in novel poses from a single image is a highly ambiguous task.
We propose ToFu, Topologically consistent Face from multi-view, a geometry inference framework that can produce topologically consistent meshes across facial identities and expressions using a volumetric representation instead of an explicit underlying 3DMM.
Second, human shape is highly correlated with gender, but existing work ignores this.
Ranked #2 on 3D Multi-Person Mesh Recovery on AGORA
Some methods produce faces that cannot be realistically animated because they do not model how wrinkles vary with expression.
To understand how people look, interact, or perform tasks, we need to quickly and accurately capture their 3D body, face, and hands together from an RGB image.
1 code implementation • 3 Sep 2019 • Bernhard Egger, William A. P. Smith, Ayush Tewari, Stefanie Wuhrer, Michael Zollhoefer, Thabo Beeler, Florian Bernard, Timo Bolkart, Adam Kortylewski, Sami Romdhani, Christian Theobalt, Volker Blanz, Thomas Vetter
In this paper, we provide a detailed survey of 3D Morphable Face Models over the 20 years since they were first proposed.
The estimation of 3D face shape from a single image must be robust to variations in lighting, head pose, expression, facial hair, makeup, and occlusions.
To address this, we introduce a unique 4D face dataset with about 29 minutes of 4D scans captured at 60 fps and synchronized audio from 12 speakers.
We use the new method, SMPLify-X, to fit SMPL-X to both controlled images and images in the wild.
Ranked #1 on 3D Human Reconstruction on Expressive hands and faces dataset (EHF) (TR V2V (mm), left hand metric)
To address this, we introduce a versatile model that learns a non-linear representation of a face using spectral convolutions on a mesh surface.
Ranked #4 on Face Alignment on FaceScape
FLAME is low-dimensional but more expressive than the FaceWarehouse model and the Basel Face Model.
Ranked #3 on Face Alignment on FaceScape
Multilinear models are widely used to represent the statistical variations of 3D human faces as they decouple shape changes due to identity and expression.
We propose a fully automatic method for fitting a 3D morphable model to single face images in arbitrary pose and lighting.
To compute a high-quality multilinear face model, the quality of the registration of the database of 3D face scans used for training is essential.
The palate model is then tested using 3D MRI from another corpus and evaluated using a high-resolution optical scan.
The resulting statistical analysis is applied to automatically generate realistic facial animations and to recognize dynamic facial expressions.
Due to the wide avail-ability of databases of high-quality data, we use the human face as the specific shape we wish to extract from corrupted data.
We show that in comparison to a global multilinear model, our model better preserves fine detail and is computationally faster, while in comparison to a localized PCA model, our model better handles variation in expression, is faster, and allows us to fix identity parameters for a given subject.
Due to the wide availability of databases of high-quality data, we use the human face as the specific shape we wish to extract from corrupted data.