An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling

4 Mar 2018Shaojie BaiJ. Zico KolterVladlen Koltun

For most deep learning practitioners, sequence modeling is synonymous with recurrent networks. Yet recent results indicate that convolutional architectures can outperform recurrent networks on tasks such as audio synthesis and machine translation... (read more)

PDF Abstract

Evaluation results from the paper


Task Dataset Model Metric name Metric value Global rank Compare
Music Modeling Nottingham TCN NLL 3.07 # 2
Music Modeling Nottingham GRU NLL 3.46 # 5
Music Modeling Nottingham RNN NLL 4.05 # 6
Music Modeling Nottingham LSTM NLL 3.29 # 3
Language Modelling Penn Treebank (Character Level) Temporal Convolutional Network Bit per Character (BPC) 1.31 # 10
Sequential Image Classification Sequential MNIST Temporal Convolutional Network Unpermuted Accuracy 99.0% # 3
Sequential Image Classification Sequential MNIST Temporal Convolutional Network Permuted Accuracy 97.2% # 2
Language Modelling WikiText-103 TCN Test perplexity 45.19 # 27