Fast-Slow Recurrent Neural Networks

NeurIPS 2017 Asier MujikaFlorian MeierAngelika Steger

Processing sequential data of variable length is a major challenge in a wide range of applications, such as speech recognition, language modeling, generative image modeling and machine translation. Here, we address this challenge by proposing a novel recurrent neural network (RNN) architecture, the Fast-Slow RNN (FS-RNN)... (read more)

PDF Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT LEADERBOARD
Language Modelling enwik8 Large FS-LSTM-4 Bit per Character (BPC) 1.25 # 15
Number of params 47M # 11
Language Modelling Hutter Prize Large FS-LSTM-4 Bit per Character (BPC) 1.245 # 8
Number of params 47M # 4
Language Modelling Hutter Prize FS-LSTM-4 Bit per Character (BPC) 1.277 # 10
Number of params 27M # 8
Language Modelling Penn Treebank (Character Level) FS-LSTM-4 Bit per Character (BPC) 1.190 # 8
Number of params 27M # 1
Language Modelling Penn Treebank (Character Level) FS-LSTM-2 Bit per Character (BPC) 1.193 # 9
Number of params 27M # 1

Methods used in the Paper


METHOD TYPE
🤖 No Methods Found Help the community by adding them if they're not listed; e.g. Deep Residual Learning for Image Recognition uses ResNet