Quo Vadis, Skeleton Action Recognition ?

In this paper, we study current and upcoming frontiers across the landscape of skeleton-based human action recognition. To study skeleton-action recognition in the wild, we introduce Skeletics-152, a curated and 3-D pose-annotated subset of RGB videos sourced from Kinetics-700, a large-scale action dataset. We extend our study to include out-of-context actions by introducing Skeleton-Mimetics, a dataset derived from the recently introduced Mimetics dataset. We also introduce Metaphorics, a dataset with caption-style annotated YouTube videos of the popular social game Dumb Charades and interpretative dance performances. We benchmark state-of-the-art models on the NTU-120 dataset and provide multi-layered assessment of the results. The results from benchmarking the top performers of NTU-120 on the newly introduced datasets reveal the challenges and domain gap induced by actions in the wild. Overall, our work characterizes the strengths and limitations of existing approaches and datasets. Via the introduced datasets, our work enables new frontiers for human action recognition.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Skeleton Based Action Recognition NTU RGB+D 120 Ensemble-top5 (MS-G3D Net + 4s Shift-GCN + VA-CNN (ResNeXt101) + 2s SDGCN + GCN-NAS (retrained)) Accuracy (Cross-Subject) 87.22% # 17
Accuracy (Cross-Setup) 88.8% # 18
Skeleton Based Action Recognition Skeletics-152 4s-ShiftGCN Accuracy (Cross-Subject) 57.01 % # 1
Skeleton Based Action Recognition Skeleton-Mimetics MS-G3D Accuracy (%) 57.37 % # 1


No methods listed for this paper. Add relevant methods here