Monocular 3D Human Pose Estimation by Generation and Ordinal Ranking

Monocular 3D human-pose estimation from static images is a challenging problem, due to the curse of dimensionality and the ill-posed nature of lifting 2D-to-3D. In this paper, we propose a Deep Conditional Variational Autoencoder based model that synthesizes diverse anatomically plausible 3D-pose samples conditioned on the estimated 2D-pose. We show that CVAE-based 3D-pose sample set is consistent with the 2D-pose and helps tackling the inherent ambiguity in 2D-to-3D lifting. We propose two strategies for obtaining the final 3D pose- (a) depth-ordering/ordinal relations to score and weight-average the candidate 3D-poses, referred to as OrdinalScore, and (b) with supervision from an Oracle. We report close to state of-the-art results on two benchmark datasets using OrdinalScore, and state-of-the-art results using the Oracle. We also show that our pipeline yields competitive results without paired image-to-3D annotations. The training and evaluation code is available at

PDF Abstract ICCV 2019 PDF ICCV 2019 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Multi-Hypotheses 3D Human Pose Estimation Human3.6M Sharma et al. Average MPJPE (mm) 46.8 # 5
Average PMPJPE (mm) 37.3 # 3
3D Human Pose Estimation Human3.6M MultiPoseNet with Oracle Average MPJPE (mm) 46.8 # 132
Monocular 3D Human Pose Estimation Human3.6M MultiPoseNet Average MPJPE (mm) 58.0 # 27
Use Video Sequence No # 1
Frames Needed 1 # 1
Need Ground Truth 2D Pose No # 1
3D Human Pose Estimation HumanEva-I Ours (Oracle) Mean Reconstruction Error (mm) 23.9 # 17