Learnable human mesh triangulation for 3D human pose and shape estimation

24 Aug 2022  ·  Sungho Chun, Sungbum Park, Ju Yong Chang ·

Compared to joint position, the accuracy of joint rotation and shape estimation has received relatively little attention in the skinned multi-person linear model (SMPL)-based human mesh reconstruction from multi-view images. The work in this field is broadly classified into two categories. The first approach performs joint estimation and then produces SMPL parameters by fitting SMPL to resultant joints. The second approach regresses SMPL parameters directly from the input images through a convolutional neural network (CNN)-based model. However, these approaches suffer from the lack of information for resolving the ambiguity of joint rotation and shape reconstruction and the difficulty of network learning. To solve the aforementioned problems, we propose a two-stage method. The proposed method first estimates the coordinates of mesh vertices through a CNN-based model from input images, and acquires SMPL parameters by fitting the SMPL model to the estimated vertices. Estimated mesh vertices provide sufficient information for determining joint rotation and shape, and are easier to learn than SMPL parameters. According to experiments using Human3.6M and MPI-INF-3DHP datasets, the proposed method significantly outperforms the previous works in terms of joint rotation and shape estimation, and achieves competitive performance in terms of joint location estimation.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
3D Human Pose Estimation Human3.6M LMT R50 224x224 Average MPJPE (mm) 30.56 # 35
Using 2D ground-truth joints No # 2
Multi-View or Monocular Multi-View # 1
MPVE (mm) 42.28 # 1
Angular Error 14.61 # 3
3D Human Pose Estimation Human3.6M LMT R152 384x384 Average MPJPE (mm) 17.59 # 3
Using 2D ground-truth joints No # 2
Multi-View or Monocular Multi-View # 1
MPVE (mm) 23.7 # 2
Angular Error 11.33 # 2
3D Human Pose Estimation MPI-INF-3DHP LMT R152 384x384 AUC 77.09 # 10
MPJPE 33.7 # 13
PCK 99.37 # 2
3D Human Pose Estimation MPI-INF-3DHP (R50-224) LMT AUC 71.57 # 13
MPJPE 45.87 # 15
PCK 96.59 # 15

Methods


No methods listed for this paper. Add relevant methods here