CHASE: Learning Convex Hull Adaptive Shift for Skeleton-based Multi-Entity Action Recognition

9 Oct 2024  ·  Yuhang Wen, Mengyuan Liu, Songtao Wu, Beichen Ding ·

Skeleton-based multi-entity action recognition is a challenging task aiming to identify interactive actions or group activities involving multiple diverse entities. Existing models for individuals often fall short in this task due to the inherent distribution discrepancies among entity skeletons, leading to suboptimal backbone optimization. To this end, we introduce a Convex Hull Adaptive Shift based multi-Entity action recognition method (CHASE), which mitigates inter-entity distribution gaps and unbiases subsequent backbones. Specifically, CHASE comprises a learnable parameterized network and an auxiliary objective. The parameterized network achieves plausible, sample-adaptive repositioning of skeleton sequences through two key components. First, the Implicit Convex Hull Constrained Adaptive Shift ensures that the new origin of the coordinate system is within the skeleton convex hull. Second, the Coefficient Learning Block provides a lightweight parameterization of the mapping from skeleton sequences to their specific coefficients in convex combinations. Moreover, to guide the optimization of this network for discrepancy minimization, we propose the Mini-batch Pair-wise Maximum Mean Discrepancy as the additional objective. CHASE operates as a sample-adaptive normalization method to mitigate inter-entity distribution discrepancies, thereby reducing data bias and improving the subsequent classifier's multi-entity action recognition performance. Extensive experiments on six datasets, including NTU Mutual 11/26, H2O, Assembly101, Collective Activity and Volleyball, consistently verify our approach by seamlessly adapting to single-entity backbones and boosting their performance in multi-entity scenarios. Our code is publicly available at https://github.com/Necolizer/CHASE .

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Benchmark
3D Action Recognition Assembly101 CHASE(CTR-GCN) Actions Top-1 28.03 # 6
Group Activity Recognition Collective Activity CHASE(CTR-GCN) Accuracy 89.61 # 4
Skeleton Based Action Recognition H2O (2 Hands and Objects) CHASE(STSA-Net) Accuracy 94.77 # 1
Human Interaction Recognition NTU RGB+D CHASE(CTR-GCN) Accuracy (Cross-Subject) 96.5 # 2
Accuracy (Cross-View) 98.8 # 2
Human Interaction Recognition NTU RGB+D 120 CHASE(CTR-GCN) Accuracy (Cross-Subject) 91.3 # 2
Accuracy (Cross-Setup) 92.3 # 2
Group Activity Recognition Volleyball CHASE(CTR-GCN) Accuracy 92.89 # 7

Methods


No methods listed for this paper. Add relevant methods here