CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimation

1 Aug 2022  ·  Zhihao LI, Jianzhuang Liu, Zhensong Zhang, Songcen Xu, Youliang Yan ·

Top-down methods dominate the field of 3D human pose and shape estimation, because they are decoupled from human detection and allow researchers to focus on the core problem. However, cropping, their first step, discards the location information from the very beginning, which makes themselves unable to accurately predict the global rotation in the original camera coordinate system. To address this problem, we propose to Carry Location Information in Full Frames (CLIFF) into this task. Specifically, we feed more holistic features to CLIFF by concatenating the cropped-image feature with its bounding box information. We calculate the 2D reprojection loss with a broader view of the full frame, taking a projection process similar to that of the person projected in the image. Fed and supervised by global-location-aware information, CLIFF directly predicts the global rotation along with more accurate articulated poses. Besides, we propose a pseudo-ground-truth annotator based on CLIFF, which provides high-quality 3D annotations for in-the-wild 2D datasets and offers crucial full supervision for regression-based methods. Extensive experiments on popular benchmarks show that CLIFF outperforms prior arts by a significant margin, and reaches the first place on the AGORA leaderboard (the SMPL-Algorithms track). The code and data are available at https://github.com/huawei-noah/noah-research/tree/master/CLIFF.

PDF Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
3D Human Pose Estimation 3DPW CLIFF+ (2DGT) PA-MPJPE 32.8 # 1
MPJPE 52.8 # 1
MPVPE 61.5 # 1
3D Human Pose Estimation 3DPW CLIFF (HR-W48) PA-MPJPE 43 # 3
MPJPE 69 # 4
MPVPE 81.2 # 2
3D Human Pose Estimation Human3.6M CLIFF (HR-W48) Average MPJPE (mm) 47.1 # 104
PA-MPJPE 32.7 # 7

Methods


No methods listed for this paper. Add relevant methods here