Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Video

Despite the recent success of single image-based 3D human pose and shape estimation methods, recovering temporally consistent and smooth 3D human motion from a video is still challenging. Several video-based methods have been proposed; however, they fail to resolve the single image-based methods' temporal inconsistency issue due to a strong dependency on a static feature of the current frame. In this regard, we present a temporally consistent mesh recovery system (TCMR). It effectively focuses on the past and future frames' temporal information without being dominated by the current static feature. Our TCMR significantly outperforms previous video-based methods in temporal consistency with better per-frame 3D pose and shape accuracy. We also release the codes. For the demo video, see https://youtu.be/WB3nTnSQDII. For the codes, see https://github.com/hongsukchoi/TCMR_RELEASE.

PDF Abstract CVPR 2021 PDF CVPR 2021 Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
3D Human Pose Estimation 3DPW TCMR (T=16 w/o H3.6M) PA-MPJPE 52.7 # 72
MPJPE 86.5 # 81
MPVPE 102.9 # 59
Acceleration Error 7.1 # 6
3D Human Pose Estimation 3DPW TCMR (T=16 w/o 3DPW) PA-MPJPE 55.8 # 84
MPJPE 95 # 97
MPVPE 111.5 # 70
Acceleration Error 7 # 5
3D Human Pose Estimation Human3.6M TCMR (T=16 w/o 3DPW) Average MPJPE (mm) 62.3 # 256
PA-MPJPE 41.1 # 60
Acceleration Error 5.3 # 9
3D Human Pose Estimation Human3.6M TCMR (T=16 w/o H3.6M) Average MPJPE (mm) 73.6 # 285
PA-MPJPE 52 # 98
Acceleration Error 3.9 # 6
3D Human Pose Estimation MPI-INF-3DHP TCMR (T=16 w/o 3DPW) MPJPE 97.4 # 62
PA-MPJPE 62.8 # 11
Acceleration Error 8 # 2
3D Human Pose Estimation MPI-INF-3DHP TCMR (T=16 w/o H3.6M) MPJPE 97.3 # 61
PA-MPJPE 63.5 # 15
Acceleration Error 8.5 # 5

Methods


No methods listed for this paper. Add relevant methods here