Domain and View-point Agnostic Hand Action Recognition

3 Mar 2021  Â·  Alberto Sabater, Iñigo Alonso, Luis Montesano, Ana C. Murillo ·

Hand action recognition is a special case of action recognition with applications in human-robot interaction, virtual reality or life-logging systems. Building action classifiers able to work for such heterogeneous action domains is very challenging. There are very subtle changes across different actions from a given application but also large variations across domains (e.g. virtual reality vs life-logging). This work introduces a novel skeleton-based hand motion representation model that tackles this problem. The framework we propose is agnostic to the application domain or camera recording view-point. When working on a single domain (intra-domain action classification) our approach performs better or similar to current state-of-the-art methods on well-known hand action recognition benchmarks. And, more importantly, when performing hand action recognition for action domains and camera perspectives which our approach has not been trained for (cross-domain action classification), our proposed framework achieves comparable performance to intra-domain state-of-the-art methods. These experiments show the robustness and generalization capabilities of our framework.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Skeleton Based Action Recognition First-Person Hand Action Benchmark TCN-Summ 1:3 Accuracy 92.9 # 1
1:1 Accuracy 95.93 # 2
3:1 Accuracy 96.76 # 1
Cross-person Accuracy 88.70 # 1
Skeleton Based Action Recognition SHREC 2017 track on 3D Hand Gesture Recognition TCN-Summ 28 gestures accuracy 91.43 # 2
14 gestures accuracy 93.57 # 3

Methods


No methods listed for this paper. Add relevant methods here