Joint Voxel and Coordinate Regression for Accurate 3D Facial Landmark Localization

28 Jan 2018  Β·  Hongwen Zhang, Qi Li, Zhenan Sun Β·

3D face shape is more expressive and viewpoint-consistent than its 2D counterpart. However, 3D facial landmark localization in a single image is challenging due to the ambiguous nature of landmarks under 3D perspective. Existing approaches typically adopt a suboptimal two-step strategy, performing 2D landmark localization followed by depth estimation. In this paper, we propose the Joint Voxel and Coordinate Regression (JVCR) method for 3D facial landmark localization, addressing it more effectively in an end-to-end fashion. First, a compact volumetric representation is proposed to encode the per-voxel likelihood of positions being the 3D landmarks. The dimensionality of such a representation is fixed regardless of the number of target landmarks, so that the curse of dimensionality could be avoided. Then, a stacked hourglass network is adopted to estimate the volumetric representation from coarse to fine, followed by a 3D convolution network that takes the estimated volume as input and regresses 3D coordinates of the face shape. In this way, the 3D structural constraints between landmarks could be learned by the neural network in a more efficient manner. Moreover, the proposed pipeline enables end-to-end training and improves the robustness and accuracy of 3D facial landmark localization. The effectiveness of our approach is validated on the 3DFAW and AFLW2000-3D datasets. Experimental results show that the proposed method achieves state-of-the-art performance in comparison with existing methods.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
3D Facial Landmark Localization 3DFAW JVCR CVGTCE 3.46 # 1
GTE 4.35 # 1
3D Facial Landmark Localization AFLW2000-3D JVCR GTE 7.28 # 1
Facial Landmark Detection AFLW2000-3D JVCR GTE 7.28 # 1