UniHCP: A Unified Model for Human-Centric Perceptions

Human-centric perceptions (e.g., pose estimation, human parsing, pedestrian detection, person re-identification, etc.) play a key role in industrial applications of visual models. While specific human-centric tasks have their own relevant semantic aspect to focus on, they also share the same underlying semantic structure of the human body. However, few works have attempted to exploit such homogeneity and design a general-propose model for human-centric tasks. In this work, we revisit a broad range of human-centric tasks and unify them in a minimalist manner. We propose UniHCP, a Unified Model for Human-Centric Perceptions, which unifies a wide range of human-centric tasks in a simplified end-to-end manner with the plain vision transformer architecture. With large-scale joint training on 33 human-centric datasets, UniHCP can outperform strong baselines on several in-domain and downstream tasks by direct evaluation. When adapted to a specific task, UniHCP achieves new SOTAs on a wide range of human-centric tasks, e.g., 69.8 mIoU on CIHP for human parsing, 86.18 mA on PA-100K for attribute prediction, 90.3 mAP on Market1501 for ReID, and 85.8 JI on CrowdHuman for pedestrian detection, performing better than specialized models tailored for each task.

PDF Abstract CVPR 2023 PDF CVPR 2023 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Pose Estimation AIC UniHCP (finetune) AP 33.6 # 2
Human Part Segmentation ATR UniHCP (FT) pACC 97.74 # 1
Pedestrian Detection Caltech UniHCP (FT) Heavy MR^-2 27.2 # 4
Human Part Segmentation CIHP UniHCP (finetune) Mean IoU 69.8 # 1
Pose Estimation COCO UniHCP (finetune) AP 76.5 # 6
Object Detection CrowdHuman (full body) UniHCP (finetune) AP 92.5 # 6
mMR 41.6 # 7
Person Re-Identification CUHK03 UniHCP (finetune) MAP 83.1 # 6
Human Part Segmentation Human3.6M UniHCP (finetune) mIoU 65.95 # 1
2D Pose Estimation Human3.6M UniHCP (finetune) EPE 6.6 # 1
Semantic Segmentation LIP val UniHCP (finetune) mIoU 63.86% # 1
Person Re-Identification Market-1501 UniHCP (finetune) mAP 90.3 # 37
Pose Estimation MPII Human Pose UniHCP (FT) PCKh-0.5 93.2 # 6
Person Re-Identification MSMT17 UniHCP (finetune) mAP 67.3 # 14
Pose Estimation OCHuman UniHCP (direct eval) Test AP 87.4 # 2
Pedestrian Attribute Recognition PA-100K UniHCP (finetune) Accuracy 86.18 # 4
Pedestrian Attribute Recognition PETA UniHCP (FT) Accuracy 88.78% # 1
Pedestrian Attribute Recognition RAPv2 UniHCP (finetune) Accuracy 82.34 # 1
Person Re-Identification SenseReID UniHCP (DE) Top-1 46 # 1