3D human pose estimation in video with temporal convolutions and semi-supervised training

In this work, we demonstrate that 3D poses in video can be effectively estimated with a fully convolutional model based on dilated temporal convolutions over 2D keypoints. We also introduce back-projection, a simple and effective semi-supervised training method that leverages unlabeled video data. We start with predicted 2D keypoints for unlabeled video, then estimate 3D poses and finally back-project to the input 2D keypoints. In the supervised setting, our fully-convolutional model outperforms the previous best result from the literature by 6 mm mean per-joint position error on Human3.6M, corresponding to an error reduction of 11%, and the model also shows significant improvements on HumanEva-I. Moreover, experiments with back-projection show that it comfortably outperforms previous state-of-the-art results in semi-supervised settings where labeled data is scarce. Code and models are available at https://github.com/facebookresearch/VideoPose3D

PDF Abstract CVPR 2019 PDF CVPR 2019 Abstract

Datasets


Results from the Paper


Ranked #11 on Weakly-supervised 3D Human Pose Estimation on Human3.6M (Number of Frames Per View metric)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Weakly-supervised 3D Human Pose Estimation Human3.6M VideoPose3D (T=243) Number of Frames Per View 243 # 11
Monocular 3D Human Pose Estimation Human3.6M VideoPose3D (T=243) Average MPJPE (mm) 46.8 # 12
Use Video Sequence Yes # 1
Frames Needed 243 # 29
Need Ground Truth 2D Pose No # 1
3D Human Pose Estimation Human3.6M VideoPose3D (T=243) Average MPJPE (mm) 46.8 # 100
Using 2D ground-truth joints No # 1
Multi-View or Monocular Monocular # 1
PA-MPJPE 36.5 # 18
3D Human Pose Estimation Human3.6M VideoPose3D (T=1) Average MPJPE (mm) 51.8 # 147
Using 2D ground-truth joints No # 1
Multi-View or Monocular Monocular # 1
PA-MPJPE 40 # 37
Weakly-supervised 3D Human Pose Estimation Human3.6M Pavllo et al. Average MPJPE (mm) 64.7 # 13
Number of Views 1 # 1
3D Annotations S1 # 1

Methods


No methods listed for this paper. Add relevant methods here