Rethinking Keypoint Representations: Modeling Keypoints and Poses as Objects for Multi-Person Human Pose Estimation

16 Nov 2021  ·  William McNally, Kanav Vats, Alexander Wong, John McPhee ·

In keypoint estimation tasks such as human pose estimation, heatmap-based regression is the dominant approach despite possessing notable drawbacks: heatmaps intrinsically suffer from quantization error and require excessive computation to generate and post-process. Motivated to find a more efficient solution, we propose to model individual keypoints and sets of spatially related keypoints (i.e., poses) as objects within a dense single-stage anchor-based detection framework. Hence, we call our method KAPAO (pronounced "Ka-Pow"), for Keypoints And Poses As Objects. KAPAO is applied to the problem of single-stage multi-person human pose estimation by simultaneously detecting human pose and keypoint objects and fusing the detections to exploit the strengths of both object representations. In experiments, we observe that KAPAO is faster and more accurate than previous methods, which suffer greatly from heatmap post-processing. The accuracy-speed trade-off is especially favourable in the practical setting when not using test-time augmentation. Source code: https://github.com/wmcnally/kapao.

PDF Abstract

Datasets


Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Pose Estimation COCO test-dev KAPAO-L AP 70.3 # 30
AP50 91.2 # 22
AP75 77.8 # 27
APL 76.8 # 23
APM 66.3 # 28
AR 77.7 # 23
Pose Estimation COCO test-dev KAPAO-M AP 68.8 # 33
AP50 90.5 # 28
AP75 76.5 # 29
APL 76 # 26
APM 64.3 # 29
AR 76.3 # 26
Pose Estimation COCO test-dev KAPAO-S AP 63.8 # 40
AP50 88.4 # 32
AP75 70.4 # 34
APL 71.7 # 32
APM 58.6 # 33
AR 71.2 # 29
Pose Estimation CrowdPose KAPAO-L AP 68.9 # 4
AP50 89.4 # 1
AP75 75.6 # 2
Test 76.6 # 1
APM 69.9 # 2
Pose Estimation CrowdPose KAPAO-M AP 67.1 # 5
AP50 88.8 # 2
AP75 73.4 # 3
Test 75.2 # 2
APM 68.1 # 3
Pose Estimation CrowdPose KAPAO-S AP 63.8 # 7
AP50 87.7 # 3
AP75 69.4 # 5
Test 72.1 # 3
APM 64.8 # 5

Methods


No methods listed for this paper. Add relevant methods here