LUVLi Face Alignment: Estimating Landmarks' Location, Uncertainty, and Visibility Likelihood

Modern face alignment methods have become quite accurate at predicting the locations of facial landmarks, but they do not typically estimate the uncertainty of their predicted locations nor predict whether landmarks are visible. In this paper, we present a novel framework for jointly predicting landmark locations, associated uncertainties of these predicted locations, and landmark visibilities. We model these as mixed random variables and estimate them using a deep network trained with our proposed Location, Uncertainty, and Visibility Likelihood (LUVLi) loss. In addition, we release an entirely new labeling of a large face alignment dataset with over 19,000 face images in a full range of head poses. Each face is manually labeled with the ground-truth locations of 68 landmarks, with the additional information of whether each landmark is unoccluded, self-occluded (due to extreme head poses), or externally occluded. Not only does our joint estimation yield accurate estimates of the uncertainty of predicted landmark locations, but it also yields state-of-the-art estimates for the landmark locations themselves on multiple standard face alignment datasets. Our method's estimates of the uncertainty of predicted landmark locations could be used to automatically identify input images on which face alignment fails, which can be critical for downstream tasks.

PDF Abstract CVPR 2020 PDF CVPR 2020 Abstract

Datasets


Introduced in the Paper:

MERL-RAV

Used in the Paper:

Multi-PIE 300W COFW WFLW AFLW-19

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Face Alignment 300W (Common) LUVLi NME 2.76 # 2
Face Alignment 300W Split 2 LUVLi NME (box) 2.24 # 3
AUC@7 (box) 68.3 # 3
NME (inter-ocular) 3.23 # 2
Face Alignment AFLW-19 LUVLi NME_diag (%, Full) 1.39 # 6
NME_diag (%, Frontal) 1.19 # 5
NME_box (%, Full) 2.28 # 6
AUC_box@0.07 (%, Full) 68.0 # 4
Face Alignment COFW-68 LUVLi NME (box) 2.75 # 4
AUC@7 (box) 60.8 # 4
Face Alignment COFW-68 LUVLi (300W-LP pretrain) NME (box) 2.57 # 3
AUC@7 (box) 63.4 # 3
Face Alignment Menpo LUVLi NME (box) 2.04 # 1
AUC (box) 71.9 # 1
Face Alignment MERL-RAV LUVLi NME (box) 1.61 # 2
AUC@7 (box) 77.08 # 2
Face Alignment WFLW LUVLi NME_inter-ocular (%, all) 4.37 # 10
AUC_inter-ocular@0.1 (%, all) 57.7 # 10
FR_inter-ocular@0.1(%, all) 3.12 # 10

Methods


No methods listed for this paper. Add relevant methods here