Neural Machine Translation in Linear Time

31 Oct 2016Nal Kalchbrenner • Lasse Espeholt • Karen Simonyan • Aaron van den Oord • Alex Graves • Koray Kavukcuoglu

We present a novel neural network for processing sequences. The ByteNet is a one-dimensional convolutional neural network that is composed of two parts, one to encode the source sequence and the other to decode the target sequence. The ByteNet decoder attains state-of-the-art performance on character-level language modelling and outperforms the previous best results obtained with recurrent networks.

Full paper


Task Dataset Model Metric name Metric value Global rank Compare
Machine Translation WMT2014 English-French ByteNet BLEU score 23.8 # 30
Machine Translation WMT2014 English-German ByteNet BLEU score 23.75 # 20
Machine Translation WMT2015 English-German ByteNet BLEU score 26.26 # 1