ASMNet: a Lightweight Deep Neural Network for Face Alignment and Pose Estimation

27 Feb 2021  ·  Ali Pourramezan Fard, Hojjat Abdollahi, Mohammad Mahoor ·

Active Shape Model (ASM) is a statistical model of object shapes that represents a target structure. ASM can guide machine learning algorithms to fit a set of points representing an object (e.g., face) onto an image. This paper presents a lightweight Convolutional Neural Network (CNN) architecture with a loss function being assisted by ASM for face alignment and estimating head pose in the wild. We use ASM to first guide the network towards learning a smoother distribution of the facial landmark points. Inspired by transfer learning, during the training process, we gradually harden the regression problem and guide the network towards learning the original landmark points distribution. We define multi-tasks in our loss function that are responsible for detecting facial landmark points as well as estimating the face pose. Learning multiple correlated tasks simultaneously builds synergy and improves the performance of individual tasks. We compare the performance of our proposed model called ASMNet with MobileNetV2 (which is about 2 times bigger than ASMNet) in both the face alignment and pose estimation tasks. Experimental results on challenging datasets show that by using the proposed ASM assisted loss function, the ASMNet performance is comparable with MobileNetV2 in the face alignment task. In addition, for face pose estimation, ASMNet performs much better than MobileNetV2. ASMNet achieves an acceptable performance for facial landmark points detection and pose estimation while having a significantly smaller number of parameters and floating-point operations compared to many CNN-based models.

PDF Abstract


Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Face Alignment 300W MobileNetV2 NME_inter-ocular (%, Full) 4.59 # 25
NME_inter-ocular (%, Common) 3.88 # 26
NME_inter-ocular (%, Challenge) 7.35 # 26
Face Alignment 300W ASMNet NME_inter-ocular (%, Full) 5.50 # 26
NME_inter-ocular (%, Common) 4.82 # 27
NME_inter-ocular (%, Challenge) 8.2 # 28
Pose Estimation 300W (Full) ASMNet MAE yaw (º) 1.62 # 2
MAE pitch (º) 1.80 # 2
MAE roll (º) 1.24 # 2
Head Pose Estimation COFW ASMNet MAE yaw (º) 2.91 # 1
MAE pitch (º) 2.72 # 1
Face Alignment WFLW ASMNet NME_inter-ocular (%, all) 10.77 # 1
Face Alignment WFLW MobileNetV2 NME_inter-ocular (%, all) 9.41 # 2
Head Pose Estimation WFLW ASMNet MAE yaw (º) 2.97 # 2
MAE roll (º) 2.21 # 2