Language Modeling with Gated Convolutional Networks

ICML 2017 Yann N. DauphinAngela FanMichael AuliDavid Grangier

The pre-dominant approach to language modeling to date is based on recurrent neural networks. Their success on this task is often linked to their ability to capture unbounded context... (read more)

PDF Abstract

Evaluation Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK COMPARE
Language Modelling One Billion Word GCNN-14 bottleneck PPL 31.9 # 10
Language Modelling WikiText-103 GCNN-8 Test perplexity 44.9 # 27
Language Modelling WikiText-103 GCNN-14 Validation perplexity - # 16
Language Modelling WikiText-103 GCNN-14 Test perplexity 37.2 # 23