Improving Neural Language Models by Segmenting, Attending, and Predicting the Future

ACL 2019 Hongyin LuoLan JiangYonatan BelinkovJames Glass

Common language models typically predict the next word given the context. In this work, we propose a method that improves language modeling by learning to align the given context and the following phrase... (read more)

PDF Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT LEADERBOARD
Language Modelling WikiText-103 Transformer-XL Large + Phrase Induction Test perplexity 17.4 # 7
Number of params 257M # 6

Methods used in the Paper


METHOD TYPE
🤖 No Methods Found Help the community by adding them if they're not listed; e.g. Deep Residual Learning for Image Recognition uses ResNet