3D Human Pose Estimation with Spatial and Temporal Transformers

Transformer architectures have become the model of choice in natural language processing and are now being introduced into computer vision tasks such as image classification, object detection, and semantic segmentation. However, in the field of human pose estimation, convolutional architectures still remain dominant. In this work, we present PoseFormer, a purely transformer-based approach for 3D human pose estimation in videos without convolutional architectures involved. Inspired by recent developments in vision transformers, we design a spatial-temporal transformer structure to comprehensively model the human joint relations within each frame as well as the temporal correlations across frames, then output an accurate 3D human pose of the center frame. We quantitatively and qualitatively evaluate our method on two popular and standard benchmark datasets: Human3.6M and MPI-INF-3DHP. Extensive experiments show that PoseFormer achieves state-of-the-art performance on both datasets. Code is available at \url{https://github.com/zczcwh/PoseFormer}

PDF Abstract ICCV 2021 PDF ICCV 2021 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Monocular 3D Human Pose Estimation Human3.6M PoseFormer (T=81) Average MPJPE (mm) 44.3 # 6
3D Human Pose Estimation Human3.6M PoseFormer (f=81, GT) Average MPJPE (mm) 31.3 # 30
Using 2D ground-truth joints Yes # 1
Multi-View or Monocular Monocular # 1
3D Human Pose Estimation Human3.6M PoseFormer (f=81) Average MPJPE (mm) 44.3 # 78
Using 2D ground-truth joints No # 1
Multi-View or Monocular Monocular # 1
3D Human Pose Estimation MPI-INF-3DHP PoseFormer (9 frames) AUC 56.4 # 12
MPJPE 77.1 # 15
PCK 88.6 # 14

Methods


No methods listed for this paper. Add relevant methods here