Multi-Instance Pose Networks: Rethinking Top-Down Pose Estimation

A key assumption of top-down human pose estimation approaches is their expectation of having a single person/instance present in the input bounding box. This often leads to failures in crowded scenes with occlusions. We propose a novel solution to overcome the limitations of this fundamental assumption. Our Multi-Instance Pose Network (MIPNet) allows for predicting multiple 2D pose instances within a given bounding box. We introduce a Multi-Instance Modulation Block (MIMB) that can adaptively modulate channel-wise feature responses for each instance and is parameter efficient. We demonstrate the efficacy of our approach by evaluating on COCO, CrowdPose, and OCHuman datasets. Specifically, we achieve 70.0 AP on CrowdPose and 42.5 AP on OCHuman test sets, a significant improvement of 2.4 AP and 6.5 AP over the prior art, respectively. When using ground truth bounding boxes for inference, MIPNet achieves an improvement of 0.7 AP on COCO, 0.9 AP on CrowdPose, and 9.1 AP on OCHuman validation sets compared to HRNet. Interestingly, when fewer, high confidence bounding boxes are used, HRNet's performance degrades (by 5 AP) on OCHuman, whereas MIPNet maintains a relatively stable performance (drop of 1 AP) for the same inputs.

PDF Abstract ICCV 2021 PDF ICCV 2021 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Pose Estimation COCO test-dev MIPNet AP 75.7 # 18
AP50 92.4 # 16
AP75 83.3 # 16
APL 81.2 # 12
APM 71.4 # 16
AR 80.5 # 16
Multi-Person Pose Estimation CrowdPose MIPNet (HRNet-W48) mAP @0.5:0.95 70.0 # 10
AP Easy 78.1 # 8
AP Medium 71.1 # 9
AP Hard 59.4 # 9
Pose Estimation CrowdPose MIPNet (HRNet-W48) AP 70.0 # 7
APM 71.1 # 2
AP Hard 59.4 # 3
Keypoint Detection MS COCO MIPNet(384x288) Validation AP 76.3 # 6
Test AP 75.7 # 7
Pose Estimation OCHuman HRNet-W48 Test AP 37.2 # 6
Validation AP 37.8 # 10
Keypoint Detection OCHuman HRNet-W48 Test AP 37.2 # 2
Validation AP 37.8 # 6
Multi-Person Pose Estimation OCHuman MIPNet (gt-bb) Validation AP 74.1 # 1
AP50 89.7 # 1
AP75 80.1 # 1
2D Human Pose Estimation OCHuman HRNet-W48 Test AP 37.2 # 3
Validation AP 37.8 # 6
Keypoint Detection OCHuman MIPNet (HRNet-W48) Test AP 42.5 # 1
Validation AP 42.0 # 1
Pose Estimation OCHuman MIPNet (HRNet-W48) Test AP 42.5 # 5
Validation AP 42.0 # 5
2D Human Pose Estimation OCHuman MIPNet (HRNet-W48) Test AP 42.5 # 2
Validation AP 42.0 # 1

Methods