Global Adaptation meets Local Generalization: Unsupervised Domain Adaptation for 3D Human Pose Estimation

ICCV 2023  ยท  Wenhao Chai, Zhongyu Jiang, Jenq-Neng Hwang, Gaoang Wang ยท

When applying a pre-trained 2D-to-3D human pose lifting model to a target unseen dataset, large performance degradation is commonly encountered due to domain shift issues. We observe that the degradation is caused by two factors: 1) the large distribution gap over global positions of poses between the source and target datasets due to variant camera parameters and settings, and 2) the deficient diversity of local structures of poses in training. To this end, we combine \textbf{global adaptation} and \textbf{local generalization} in \textit{PoseDA}, a simple yet effective framework of unsupervised domain adaptation for 3D human pose estimation. Specifically, global adaptation aims to align global positions of poses from the source domain to the target domain with a proposed global position alignment (GPA) module. And local generalization is designed to enhance the diversity of 2D-3D pose mapping with a local pose augmentation (LPA) module. These modules bring significant performance improvement without introducing additional learnable parameters. In addition, we propose local pose augmentation (LPA) to enhance the diversity of 3D poses following an adversarial training scheme consisting of 1) a augmentation generator that generates the parameters of pre-defined pose transformations and 2) an anchor discriminator to ensure the reality and quality of the augmented data. Our approach can be applicable to almost all 2D-3D lifting models. \textit{PoseDA} achieves 61.3 mm of MPJPE on MPI-INF-3DHP under a cross-dataset evaluation setup, improving upon the previous state-of-the-art method by 10.2\%.

PDF Abstract ICCV 2023 PDF ICCV 2023 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Cross-domain 3D Human Pose Estimation 3DPW PoseDA PA-MPJPE 55.3 # 1
MPJPE 87.7 # 1
3D Human Pose Estimation in Limited Data Human3.6M PoseDA MPJPE 49.9 # 1
PA-MPJPE 34.2 # 1
3D Human Pose Estimation MPI-INF-3DHP PoseDA AUC 62.5 # 23
MPJPE 61.3 # 25
PCK 92.1 # 25
Cross-domain 3D Human Pose Estimation MPI-INF-3DHP PoseDA MPJPE 61.36 # 1
PCK 92.05 # 1
AUC 0.6252 # 1

Methods