Improving the Performance of Unimodal Dynamic Hand-Gesture Recognition with Multimodal Training

CVPR 2019 Mahdi AbavisaniHamid Reza Vaezi JozeVishal M. Patel

We present an efficient approach for leveraging the knowledge from multiple modalities in training unimodal 3D convolutional neural networks (3D-CNNs) for the task of dynamic hand gesture recognition. Instead of explicitly combining multimodal information, which is commonplace in many state-of-the-art methods, we propose a different framework in which we embed the knowledge of multiple modalities in individual networks so that each unimodal network can achieve an improved performance... (read more)

PDF Abstract
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK COMPARE
Action Recognition In Videos EgoGesture MTUT Accuracy 93.87 # 2
Hand-Gesture Recognition EgoGesture MTUT Accuracy 93.87 # 2
Hand-Gesture Recognition NVGesture MTUT Accuracy 86.93 # 1
Action Recognition In Videos VIVA Hand Gestures Dataset MTUT Accuracy 86.08 # 1
Hand-Gesture Recognition VIVA Hand Gestures Dataset MTUT Accuracy 86.08 # 1