Shape Preserving Facial Landmarks with Graph Attention Networks

13 Oct 2022  ·  Andrés Prados-Torreblanca, José M. Buenaposada, Luis Baumela ·

Top-performing landmark estimation algorithms are based on exploiting the excellent ability of large convolutional neural networks (CNNs) to represent local appearance. However, it is well known that they can only learn weak spatial relationships. To address this problem, we propose a model based on the combination of a CNN with a cascade of Graph Attention Network regressors. To this end, we introduce an encoding that jointly represents the appearance and location of facial landmarks and an attention mechanism to weigh the information according to its reliability. This is combined with a multi-task approach to initialize the location of graph nodes and a coarse-to-fine landmark description scheme. Our experiments confirm that the proposed model learns a global representation of the structure of the face, achieving top performance in popular benchmarks on head pose and landmark estimation. The improvement provided by our model is most significant in situations involving large changes in the local appearance of landmarks.

PDF Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Facial Landmark Detection 300W SPIGA (Inter-ocular Norm) NME 2.99 # 1
Face Alignment 300W SPIGA NME_inter-ocular (%, Full) 2.99 # 5
NME_inter-ocular (%, Common) 2.59 # 4
NME_inter-ocular (%, Challenge) 4.66 # 6
NME_inter-pupil (%, Full) 4.20 # 7
NME_inter-pupil (%, Common) 3.59 # 7
NME_inter-pupil (%, Challenge) 6.73 # 6
Face Alignment 300W (Common) SPIGA NME 2.59 # 1
Pose Estimation 300W (Full) SPIGA MAE mean (º) 1.29 # 1
MAE yaw (º) 1.41 # 1
MAE pitch (º) 1.70 # 1
MAE roll (º) 0.77 # 1
Face Alignment 300W Split 2 SPIGA NME (box) 2.03 # 1
AUC@7 (box) 71.0 # 1
NME (inter-ocular) 3.43 # 3
AUC@8 (inter-ocular) 57.27 # 1
FR@8 (inter-ocular) 0.67 # 1
Face Alignment COFW-68 SPIGA NME (box) 2.52 # 2
AUC@7 (box) 64.1 # 2
NME (inter-ocular) 3.93 # 1
Pose Estimation MERL-RAV SPIGA MAE mean (º) 2.39 # 1
MAE yaw (º) 3.23 # 1
MAE pitch (º) 2.24 # 1
MAE roll (º) 1.71 # 1
Face Alignment MERL-RAV SPIGA NME (box) 1.51 # 1
AUC@7 (box) 78.47 # 1
Head Pose Estimation WFLW SPIGA MAE mean (º) 1.52 # 1
MAE yaw (º) 1.78 # 1
MAE pitch (º) 1.86 # 1
MAE roll (º) 0.93 # 1
Face Alignment WFLW SPIGA NME_inter-ocular (%, all) 4.06 # 3
AUC_inter-ocular@0.1 (%, all) 60.56 # 3
FR_inter-ocular@0.1(%, all) 2.08 # 3

Methods


No methods listed for this paper. Add relevant methods here