Recurrent Highway Networks with Grouped Auxiliary Memory

IEEE Access 2019 Wei Luo ; Feng Yu

Recurrent neural networks (RNNs) are challenging to train, let alone those with deep spatial structures. Architectures built upon highway connections such as Recurrent Highway Network (RHN) were developed to allow larger step-to-step transition depth, leading to more expressive models... (read more)

PDF

Evaluation Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK COMPARE
Stock Trend Prediction FI-2010 BL-GAM-RHN-7 F1 (H50) 0.8088 # 1
Stock Trend Prediction FI-2010 BL-GAM-RHN-7 Accuracy (H50) 0.8202 # 1
Language Modelling Penn Treebank (Character Level) GAM-RHN-5 Bit per Character (BPC) 1.147 # 1
Language Modelling Penn Treebank (Character Level) GAM-RHN-5 Number of params 16.0M # 1
Sequential Image Classification Sequential MNIST GAM-RHN-1 Permuted Accuracy 96.8% # 2
Language Modelling Text8 GAM-RHN-10 Bit per Character (BPC) 1.157 # 7
Language Modelling Text8 GAM-RHN-10 Number of params 44.7M # 1