Face Alignment using a 3D Deeply-initialized Ensemble of Regression Trees

5 Feb 2019  ·  Roberto Valle, José M. Buenaposada, Antonio Valdés, Luis Baumela ·

Face alignment algorithms locate a set of landmark points in images of faces taken in unrestricted situations. State-of-the-art approaches typically fail or lose accuracy in the presence of occlusions, strong deformations, large pose variations and ambiguous configurations. In this paper we present 3DDE, a robust and efficient face alignment algorithm based on a coarse-to-fine cascade of ensembles of regression trees. It is initialized by robustly fitting a 3D face model to the probability maps produced by a convolutional neural network. With this initialization we address self-occlusions and large face rotations. Further, the regressor implicitly imposes a prior face shape on the solution, addressing occlusions and ambiguous face configurations. Its coarse-to-fine structure tackles the combinatorial explosion of parts deformation. In the experiments performed, 3DDE improves the state-of-the-art in 300W, COFW, AFLW and WFLW data sets. Finally, we perform cross-dataset experiments that reveal the existence of a significant data set bias in these benchmarks.

PDF Abstract

Results from the Paper


Ranked #2 on Face Alignment on 300W Split 2 (NME (inter-ocular) metric)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Facial Landmark Detection 300W 3DDE (Inter-ocular Norm) NME 3.13 # 4
Face Alignment 300W 3DDE NME_inter-ocular (%, Full) 3.13 # 15
NME_inter-ocular (%, Common) 2.69 # 11
NME_inter-ocular (%, Challenge) 4.92 # 21
NME_inter-pupil (%, Full) 4.39 # 12
NME_inter-pupil (%, Common) 3.73 # 10
NME_inter-pupil (%, Challenge) 7.10 # 13
Face Alignment 300W Split 2 3DDE NME (inter-ocular) 3.73 # 2
AUC@8 (inter-ocular) 53.94 # 2
FR@8 (inter-ocular) 2.33 # 3
Facial Landmark Detection AFLW-Full 3DDE (Box height Norm, 19 landmarks - no earlobs) Mean NME 2.01 # 3
Face Alignment COFW 3DDE (Inter-pupil Norm) NME (inter-pupil) 5.11% # 5
Recall at 80% precision (Landmarks Visibility) 63.89 # 2
Face Alignment WFLW 3DDE NME (inter-ocular) 4.68 # 26
AUC@10 (inter-ocular) 55.44 # 20
FR@10 (inter-ocular) 5.04 # 21

Methods


No methods listed for this paper. Add relevant methods here