Trellis Networks for Sequence Modeling

ICLR 2019 Shaojie BaiJ. Zico KolterVladlen Koltun

We present trellis networks, a new architecture for sequence modeling. On the one hand, a trellis network is a temporal convolutional network with special structure, characterized by weight tying across depth and direct injection of the input into deep layers... (read more)

PDF Abstract

Evaluation results from the paper


Task Dataset Model Metric name Metric value Global rank Compare
Language Modelling Penn Treebank (Character Level) Trellis Network Bit per Character (BPC) 1.158 # 1
Language Modelling Penn Treebank (Character Level) Trellis Network Number of params 13.4M # 1
Language Modelling Penn Treebank (Word Level) Trellis Network Test perplexity 54.19 # 12
Sequential Image Classification Sequential CIFAR-10 Trellis Network Unpermuted Accuracy 73.42% # 1
Sequential Image Classification Sequential MNIST Trellis Network Unpermuted Accuracy 99.2% # 1
Sequential Image Classification Sequential MNIST Trellis Network Permuted Accuracy 98.13% # 1
Language Modelling WikiText-103 Trellis Network Test perplexity 29.19 # 7