Ray3D: ray-based 3D human pose estimation for monocular absolute 3D localization

CVPR 2022  ·  Yu Zhan, Fenghai Li, Renliang Weng, Wongun Choi ·

In this paper, we propose a novel monocular ray-based 3D (Ray3D) absolute human pose estimation with calibrated camera. Accurate and generalizable absolute 3D human pose estimation from monocular 2D pose input is an ill-posed problem. To address this challenge, we convert the input from pixel space to 3D normalized rays. This conversion makes our approach robust to camera intrinsic parameter changes. To deal with the in-the-wild camera extrinsic parameter variations, Ray3D explicitly takes the camera extrinsic parameters as an input and jointly models the distribution between the 3D pose rays and camera extrinsic parameters. This novel network design is the key to the outstanding generalizability of Ray3D approach. To have a comprehensive understanding of how the camera intrinsic and extrinsic parameter variations affect the accuracy of absolute 3D key-point localization, we conduct in-depth systematic experiments on three single person 3D benchmarks as well as one synthetic benchmark. These experiments demonstrate that our method significantly outperforms existing state-of-the-art models. Our code and the synthetic dataset are available at https://github.com/YxZhxn/Ray3D .

PDF Abstract CVPR 2022 PDF CVPR 2022 Abstract

Results from the Paper


 Ranked #1 on Monocular 3D Human Pose Estimation on Human3.6M (Use Video Sequence metric)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Monocular 3D Human Pose Estimation Human3.6M Ray3D Use Video Sequence Yes # 1
Frames Needed 9 # 24
Need Ground Truth 2D Pose No # 1
3D Human Pose Estimation Human3.6M Ray3D (T=9 GT) Average MPJPE (mm) 34.4 # 38
Using 2D ground-truth joints Yes # 1
Multi-View or Monocular Monocular # 1
3D Human Pose Estimation Human3.6M Ray3D (T=9 CPN) Average MPJPE (mm) 49.7 # 119
Using 2D ground-truth joints No # 1
Multi-View or Monocular Monocular # 1
3D Human Pose Estimation Human3.6M Ray3D (T=9 CPN H36M+HEva+3DHP) Average MPJPE (mm) 84.4 # 238
Using 2D ground-truth joints No # 1
Multi-View or Monocular Monocular # 1
3D Human Pose Estimation MPI-INF-3DHP Ray3D (T=9 CPN H36M+HEva+3DHP) MPJPE 46.6 # 3

Methods


No methods listed for this paper. Add relevant methods here