CanonPose: Self-Supervised Monocular 3D Human Pose Estimation in the Wild

Human pose estimation from single images is a challenging problem in computer vision that requires large amounts of labeled training data to be solved accurately. Unfortunately, for many human activities (\eg outdoor sports) such training data does not exist and is hard or even impossible to acquire with traditional motion capture systems. We propose a self-supervised approach that learns a single image 3D pose estimator from unlabeled multi-view data. To this end, we exploit multi-view consistency constraints to disentangle the observed 2D pose into the underlying 3D pose and camera rotation. In contrast to most existing methods, we do not require calibrated cameras and can therefore learn from moving cameras. Nevertheless, in the case of a static camera setup, we present an optional extension to include constant relative camera rotations over multiple views into our framework. Key to the success are new, unbiased reconstruction objectives that mix information across views and training samples. The proposed approach is evaluated on two benchmark datasets (Human3.6M and MPII-INF-3DHP) and on the in-the-wild SkiPose dataset.

PDF Abstract CVPR 2021 PDF CVPR 2021 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
3D Human Pose Estimation Human3.6M CanonPose Average MPJPE (mm) 74.3 # 238
Weakly-supervised 3D Human Pose Estimation Human3.6M CanonPose Average MPJPE (mm) 74.3 # 17
Number of Views 1 # 1
Number of Frames Per View 1 # 1
3D Annotations No # 1
3D Human Pose Estimation MPI-INF-3DHP CanonPose MPJPE 104 # 47
PCK 77 # 49
3D Human Pose Estimation SkiPose CanonPose MPJPE 128.1 # 3
P-MPJPE 89.6 # 1
PCK 67.1 # 1
CPS 108.7 # 1

Methods


No methods listed for this paper. Add relevant methods here