SEFD: Learning to Distill Complex Pose and Occlusion
This paper addresses the problem of three-dimensional (3D) human mesh estimation in complex poses and occluded situations. Although many improvements have been made in 3D human mesh estimation using the two-dimensional (2D) pose with occlusion between humans, occlusion from complex poses and other objects remains a consistent problem. Therefore, we propose the novel Skinned Multi-Person Linear (SMPL) Edge Feature Distillation (SEFD) that demonstrates robustness to complex poses and occlusions, without increasing the number of parameters compared to the baseline model. The model generates an SMPL overlapping edge similar to the ground truth that contains target person boundary and occlusion information, performing subsequent feature distillation in a simple edge map. We also perform experiments on various benchmarks and exhibit fidelity both qualitatively and quantitatively. Extensive experiments prove that our method outperforms the state-of-the-art method by 2.8% in MPJPE and 1.9% in MPVPE on a benchmark 3DPW dataset in the presence of domain gap. Also, our method is superior in 3DPW-OCC, 3DPW-PC, RH-Dataset, OCHuman, CrowdPose, and LSP dataset in which occlusion, complex pose, and domain gap exist. The code and occlusion & complex pose annotation will be available at https: //anonymous.4open.science/r/SEFD-B7F8/
PDF Abstract
MS COCO
3DPW
LSP
CrowdPose
OCHuman
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
|---|---|---|---|---|---|---|
| 3D Human Pose Estimation | 3DPW | SEFD_GT | PA-MPJPE | 43.79 | # 27 | |
| MPJPE | 64.75 | # 7 | ||||
| MPVPE | 78.36 | # 12 | ||||
| 3D Human Pose Estimation | 3DPW | SEFD | PA-MPJPE | 49.39 | # 50 | |
| MPJPE | 77.37 | # 52 | ||||
| MPVPE | 92.60 | # 40 | ||||
| 2D Human Pose Estimation | OCHuman | SEFD | Test AP | 44.1 | # 1 |
