Recognizing Surgical Activities with Recurrent Neural Networks

We apply recurrent neural networks to the task of recognizing surgical activities from robot kinematics. Prior work in this area focuses on recognizing short, low-level activities, or gestures, and has been based on variants of hidden Markov models and conditional random fields. In contrast, we work on recognizing both gestures and longer, higher-level activites, or maneuvers, and we model the mapping from kinematics to gestures/maneuvers with recurrent neural networks. To our knowledge, we are the first to apply recurrent neural networks to this task. Using a single model and a single set of hyperparameters, we match state-of-the-art performance for gesture recognition and advance state-of-the-art performance for maneuver recognition, in terms of both accuracy and edit distance. Code is available at .

PDF Abstract


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Surgical Skills Evaluation JIGSAWS Bidir. LSTM Accuracy 0.833 # 2
Edit Distance 14.6 # 1
Surgical Skills Evaluation MISTIC-SIL Bidir. LSTM Accuracy 0.895 # 1
Edit Distance 19.5 # 1


No methods listed for this paper. Add relevant methods here