img2pose: Face Alignment and Detection via 6DoF, Face Pose Estimation

We propose real-time, six degrees of freedom (6DoF), 3D face pose estimation without face detection or landmark localization. We observe that estimating the 6DoF rigid transformation of a face is a simpler problem than facial landmark detection, often used for 3D face alignment. In addition, 6DoF offers more information than face bounding box labels. We leverage these observations to make multiple contributions: (a) We describe an easily trained, efficient, Faster R-CNN--based model which regresses 6DoF pose for all faces in the photo, without preliminary face detection. (b) We explain how pose is converted and kept consistent between the input photo and arbitrary crops created while training and evaluating our model. (c) Finally, we show how face poses can replace detection bounding box training labels. Tests on AFLW2000-3D and BIWI show that our method runs at real-time and outperforms state of the art (SotA) face pose estimators. Remarkably, our method also surpasses SotA models of comparable complexity on the WIDER FACE detection benchmark, despite not been optimized on bounding box labels.

PDF Abstract CVPR 2021 PDF CVPR 2021 Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Head Pose Estimation AFLW2000 RetinaFace R-50 (5 points) MAE 4.839 # 14
MAE_t 0.114 # 1
Head Pose Estimation AFLW2000 img2pose MAE 3.913 # 8
MAE_t 0.099 # 2
Geodesic Error (GE) 6.41 # 3
Head Pose Estimation BIWI RetinaFace R-50 (5 points) MAE (trained with other data) 4.578 # 12
Head Pose Estimation BIWI img2pose MAE (trained with other data) 3.786 # 5
Geodesic Error (GE) 7.10 # 1
MAE-aligned (trained with other data) 3.4 # 4
Geodesic Error - aligned (GE) 6.23 # 4
Face Detection WIDER Face (Easy) img2pose AP 0.9 # 25
Face Detection WIDER Face (Hard) img2pose AP 0.839 # 24
Face Detection WIDER Face (Medium) img2pose AP 0.890 # 26


No methods listed for this paper. Add relevant methods here