Subpixel Heatmap Regression for Facial Landmark Localization

3 Nov 2021  ·  Adrian Bulat, Enrique Sanchez, Georgios Tzimiropoulos ·

Deep Learning models based on heatmap regression have revolutionized the task of facial landmark localization with existing models working robustly under large poses, non-uniform illumination and shadows, occlusions and self-occlusions, low resolution and blur. However, despite their wide adoption, heatmap regression approaches suffer from discretization-induced errors related to both the heatmap encoding and decoding process. In this work we show that these errors have a surprisingly large negative impact on facial alignment accuracy. To alleviate this problem, we propose a new approach for the heatmap encoding and decoding process by leveraging the underlying continuous distribution. To take full advantage of the newly proposed encoding-decoding mechanism, we also introduce a Siamese-based training that enforces heatmap consistency across various geometric image transformations. Our approach offers noticeable gains across multiple datasets setting a new state-of-the-art result in facial landmark localization. Code alongside the pretrained models will be made available at https://www.adrianbulat.com/face-alignment

PDF Abstract
No code implementations yet. Submit your code now

Results from the Paper


 Ranked #1 on Face Alignment on WFW (Extra Data) (using extra training data)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Face Alignment 300W SHR-FAN NME_inter-ocular (%, Full) 2.94 # 8
NME_inter-ocular (%, Common) 2.61 # 8
NME_inter-ocular (%, Challenge) 4.13 # 2
Face Alignment 300W Split 2 (300W-LP) SH-FAN NME (inter-ocular) 2.94 # 1
NME (bbox) 2.04 # 1
AUC@7 (bbox) 71.1 # 1
Face Alignment AFLW-19 SHR-FAN NME_diag (%, Full) 1.31 # 5
NME_diag (%, Frontal) 1.12 # 4
NME_box (%, Full) 2.14 # 6
AUC_box@0.07 (%, Full) 70.0 # 4
Face Alignment COFW SH-FAN NME (inter-ocular) 3.02% # 2
Face Alignment COFW-68 (300WLP) SH-FAN NME (box) 2.47 # 1
AUC@7 64.9 # 1
Face Alignment WFW (Extra Data) SH-FAN NME (inter-ocular) 3.72 # 1
AUC@10 (inter-ocular) 63.1 # 1
FR@10 (inter-ocular) 1.55 # 1

Methods