Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network

ECCV 2018  ·  Yao Feng, Fan Wu, Xiaohu Shao, Yan-Feng Wang, Xi Zhou ·

We propose a straightforward method that simultaneously reconstructs the 3D facial structure and provides dense alignment. To achieve this, we design a 2D representation called UV position map which records the 3D shape of a complete face in UV space, then train a simple Convolutional Neural Network to regress it from a single 2D image. We also integrate a weight mask into the loss function during training to improve the performance of the network. Our method does not rely on any prior face model, and can reconstruct full facial geometry along with semantic meaning. Meanwhile, our network is very light-weighted and spends only 9.8ms to process an image, which is extremely faster than previous works. Experiments on multiple challenging datasets show that our method surpasses other state-of-the-art methods on both reconstruction and alignment tasks by a large margin.

PDF Abstract ECCV 2018 PDF ECCV 2018 Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Face Alignment AFLW2000-3D PRN Mean NME 3.62% # 8
3D Face Reconstruction AFLW2000-3D PRN Mean NME 3.9625% # 5
Face Alignment AFLW-LFPA FPN Mean NME 2.93% # 1
3D Face Reconstruction Florence PRN Mean NME 3.7551% # 1
3D Face Reconstruction NoW Benchmark PRNet Mean Reconstruction Error (mm) 1.98 # 16
Stdev Reconstruction Error (mm) 1.88 # 15
Median Reconstruction Error 1.50 # 15
3D Face Reconstruction REALY PRNet @nose 1.923 (±0.518) # 13
@mouth 1.838 (±0.637) # 12
@forehead 2.429 (±0.588) # 14
@cheek 1.863 (±0.698) # 19
all 2.013 # 12
3D Face Reconstruction REALY (side-view) PRNet @nose 1.868 (±0.510) # 10
all 2.032 # 10
@mouth 1.856 (±0.607) # 9
@forehead 2.445 (±0.570) # 9
@cheek 1.960 (±0.731) # 15
3D Face Reconstruction Stirling-HQ (FG2018 3D face reconstruction challenge) PRNet Mean Reconstruction Error (mm) 2.06 # 4
3D Face Reconstruction Stirling-LQ (FG2018 3D face reconstruction challenge) PRNet Mean Reconstruction Error (mm) 2.38 # 4


No methods listed for this paper. Add relevant methods here