HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation

Model-based 3D pose and shape estimation methods reconstruct a full 3D mesh for the human body by estimating several parameters. However, learning the abstract parameters is a highly non-linear process and suffers from image-model misalignment, leading to mediocre model performance. In contrast, 3D keypoint estimation methods combine deep CNN network with the volumetric representation to achieve pixel-level localization accuracy but may predict unrealistic body structure. In this paper, we address the above issues by bridging the gap between body mesh estimation and 3D keypoint estimation. We propose a novel hybrid inverse kinematics solution (HybrIK). HybrIK directly transforms accurate 3D joints to relative body-part rotations for 3D body mesh reconstruction, via the twist-and-swing decomposition. The swing rotation is analytically solved with 3D joints, and the twist rotation is derived from the visual cues through the neural network. We show that HybrIK preserves both the accuracy of 3D pose and the realistic body structure of the parametric human model, leading to a pixel-aligned 3D body mesh and a more accurate 3D pose than the pure 3D keypoint estimation methods. Without bells and whistles, the proposed method surpasses the state-of-the-art methods by a large margin on various 3D human pose and shape benchmarks. As an illustrative example, HybrIK outperforms all the previous methods by 13.2 mm MPJPE and 21.9 mm PVE on 3DPW dataset. Our code is available at https://github.com/Jeff-sjtu/HybrIK.

PDF Abstract CVPR 2021 PDF CVPR 2021 Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
3D Human Pose Estimation 3DPW HybrIK PA-MPJPE 45.0 # 35
MPJPE 74.1 # 34
MPVPE 86.5 # 28
3D Human Pose Estimation EMDB HybrIK Average MPJPE (mm) 103.037 # 3
Average MPJPE-PA (mm) 65.5935 # 3
Average MVE (mm) 122.193 # 4
Average MVE-PA (mm) 80.3678 # 1
Average MPJAE (deg) 24.5174 # 2
Average MPJAE-PA (deg) 23.0704 # 5
Jitter (10m/s^3) 49.2068 # 1
3D Human Pose Estimation Human3.6M HybrIK Average MPJPE (mm) 54.4 # 213
PA-MPJPE 33.6 # 17
3D Human Pose Estimation MPI-INF-3DHP HybrIK AUC 46.9 # 50
MPJPE 91.0 # 44
PCK 87.5 # 37


No methods listed for this paper. Add relevant methods here