A Simple Way to Initialize Recurrent Networks of Rectified Linear Units

3 Apr 2015Quoc V. LeNavdeep JaitlyGeoffrey E. Hinton

Learning long term dependencies in recurrent networks is difficult due to vanishing and exploding gradients. To overcome this difficulty, researchers have developed sophisticated optimization techniques and network architectures... (read more)

PDF Abstract

Evaluation results from the paper


Task Dataset Model Metric name Metric value Global rank Compare
Sequential Image Classification Sequential MNIST iRNN Unpermuted Accuracy 97% # 5
Sequential Image Classification Sequential MNIST iRNN Permuted Accuracy 82% # 7