SkeleMotion: A New Representation of Skeleton Joint Sequences Based on Motion Information for 3D Action Recognition

Due to the availability of large-scale skeleton datasets, 3D human action recognition has recently called the attention of computer vision community. Many works have focused on encoding skeleton data as skeleton image representations based on spatial structure of the skeleton joints, in which the temporal dynamics of the sequence is encoded as variations in columns and the spatial structure of each frame is represented as rows of a matrix. To further improve such representations, we introduce a novel skeleton image representation to be used as input of Convolutional Neural Networks (CNNs), named SkeleMotion. The proposed approach encodes the temporal dynamics by explicitly computing the magnitude and orientation values of the skeleton joints. Different temporal scales are employed to compute motion values to aggregate more temporal dynamics to the representation making it able to capture longrange joint interactions involved in actions as well as filtering noisy motion values. Experimental results demonstrate the effectiveness of the proposed representation on 3D action recognition outperforming the state-of-the-art on NTU RGB+D 120 dataset.

PDF Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Skeleton Based Action Recognition NTU RGB+D Skelemotion + Yang et al. Accuracy (CV) 84.7 # 100
Accuracy (CS) 76.5 # 104
Action Recognition NTU RGB+D Skelemotion + Yang et al. (Skeleton only) Accuracy (CS) 76.5 # 24
Accuracy (CV) 84.7 # 21
Action Recognition NTU RGB+D 120 Skelemotion + Yang et al. (skeleton only) Accuracy (Cross-Subject) 67.7 # 16
Accuracy (Cross-Setup) 66.9 # 17
Skeleton Based Action Recognition NTU RGB+D 120 SkeleMotion [Magnitude-Orientation (TSA)] Accuracy (Cross-Subject) 62.9% # 62
Accuracy (Cross-Setup) 63.0% # 61
Skeleton Based Action Recognition NTU RGB+D 120 SkeleMotion + Yang et al. (2018) Accuracy (Cross-Subject) 67.7% # 59
Accuracy (Cross-Setup) 66.9% # 57

Methods


No methods listed for this paper. Add relevant methods here