Learning Dynamical Human-Joint Affinity for 3D Pose Estimation in Videos

15 Sep 2021  ·  Junhao Zhang, Yali Wang, Zhipeng Zhou, Tianyu Luan, Zhe Wang, Yu Qiao ·

Graph Convolution Network (GCN) has been successfully used for 3D human pose estimation in videos. However, it is often built on the fixed human-joint affinity, according to human skeleton. This may reduce adaptation capacity of GCN to tackle complex spatio-temporal pose variations in videos. To alleviate this problem, we propose a novel Dynamical Graph Network (DG-Net), which can dynamically identify human-joint affinity, and estimate 3D pose by adaptively learning spatial/temporal joint relations from videos. Different from traditional graph convolution, we introduce Dynamical Spatial/Temporal Graph convolution (DSG/DTG) to discover spatial/temporal human-joint affinity for each video exemplar, depending on spatial distance/temporal movement similarity between human joints in this video. Hence, they can effectively understand which joints are spatially closer and/or have consistent motion, for reducing depth ambiguity and/or motion uncertainty when lifting 2D pose to 3D pose. We conduct extensive experiments on three popular benchmarks, e.g., Human3.6M, HumanEva-I, and MPI-INF-3DHP, where DG-Net outperforms a number of recent SOTA approaches with fewer input frames and model size.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
3D Human Pose Estimation Human3.6M DG-Net (T=4 GTi) Average MPJPE (mm) 31.2 # 38
Using 2D ground-truth joints Yes # 2
Multi-View or Monocular Monocular # 1
3D Human Pose Estimation Human3.6M DG-Net (T=4) Average MPJPE (mm) 45.3 # 110
3D Human Pose Estimation HumanEva-I DG-Net (T=4) Mean Reconstruction Error (mm) 19.5 # 10
Pose Estimation Leeds Sports Poses DG-Net (T=4) PCK 87.5% # 14
3D Human Pose Estimation MPI-INF-3DHP DG-Net (T=4) AUC 53.8 # 36
MPJPE 76 # 29