MotionAGFormer: Enhancing 3D Human Pose Estimation with a Transformer-GCNFormer Network

25 Oct 2023  Â·  Soroush Mehraban, Vida Adeli, Babak Taati ·

Recent transformer-based approaches have demonstrated excellent performance in 3D human pose estimation. However, they have a holistic view and by encoding global relationships between all the joints, they do not capture the local dependencies precisely. In this paper, we present a novel Attention-GCNFormer (AGFormer) block that divides the number of channels by using two parallel transformer and GCNFormer streams. Our proposed GCNFormer module exploits the local relationship between adjacent joints, outputting a new representation that is complementary to the transformer output. By fusing these two representation in an adaptive way, AGFormer exhibits the ability to better learn the underlying 3D structure. By stacking multiple AGFormer blocks, we propose MotionAGFormer in four different variants, which can be chosen based on the speed-accuracy trade-off. We evaluate our model on two popular benchmark datasets: Human3.6M and MPI-INF-3DHP. MotionAGFormer-B achieves state-of-the-art results, with P1 errors of 38.4mm and 16.2mm, respectively. Remarkably, it uses a quarter of the parameters and is three times more computationally efficient than the previous leading model on Human3.6M dataset. Code and models are available at https://github.com/TaatiTeam/MotionAGFormer.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Classification Full-body Parkinson’s disease dataset MotionAGFormer F1-score (weighted) 0.42 # 6
Monocular 3D Human Pose Estimation Human3.6M MotionAGFormer-L Average MPJPE (mm) 38.4 # 2
Use Video Sequence Yes # 1
Frames Needed 243 # 33
Need Ground Truth 2D Pose No # 1
2D detector SH # 1
Monocular 3D Human Pose Estimation Human3.6M MotionAGFormer-XS Average MPJPE (mm) 45.1 # 15
Use Video Sequence Yes # 1
Frames Needed 27 # 27
Need Ground Truth 2D Pose No # 1
2D detector SH # 1
Monocular 3D Human Pose Estimation Human3.6M MotionAGFormer-S Average MPJPE (mm) 42.5 # 11
Use Video Sequence Yes # 1
Frames Needed 81 # 29
Need Ground Truth 2D Pose No # 1
2D detector SH # 1
Monocular 3D Human Pose Estimation Human3.6M MotionAGFormer-B Average MPJPE (mm) 38.4 # 2
Use Video Sequence Yes # 1
Frames Needed 243 # 33
Need Ground Truth 2D Pose No # 1
2D detector SH # 1
3D Human Pose Estimation MPI-INF-3DHP MotionAGFormer-B (T=81) AUC 84.2 # 4
MPJPE 18.2 # 4
PCK 98.3 # 6
3D Human Pose Estimation MPI-INF-3DHP MotionAGFormer-XS (T=27) AUC 83.5 # 6
MPJPE 19.2 # 5
PCK 98.2 # 8
3D Human Pose Estimation MPI-INF-3DHP MotionAGFormer-S (T=81) AUC 84.5 # 3
MPJPE 17.1 # 3
PCK 98.3 # 6
3D Human Pose Estimation MPI-INF-3DHP MotionAGFormer-L (T=81) AUC 85.3 # 2
MPJPE 16.2 # 1
PCK 98.2 # 8

Methods


No methods listed for this paper. Add relevant methods here