Tracking the World State with Recurrent Entity Networks

12 Dec 2016  ·  Mikael Henaff, Jason Weston, Arthur Szlam, Antoine Bordes, Yann Lecun ·

We introduce a new model, the Recurrent Entity Network (EntNet). It is equipped with a dynamic long-term memory which allows it to maintain and update a representation of the state of the world as it receives new data. For language understanding tasks, it can reason on-the-fly as it reads text, not just when it is required to answer a question or respond as is the case for a Memory Network (Sukhbaatar et al., 2015). Like a Neural Turing Machine or Differentiable Neural Computer (Graves et al., 2014; 2016) it maintains a fixed size memory and can learn to perform location and content-based read and write operations. However, unlike those models it has a simple parallel architecture in which several memory locations can be updated simultaneously. The EntNet sets a new state-of-the-art on the bAbI tasks, and is the first method to solve all the tasks in the 10k training examples setting. We also demonstrate that it can solve a reasoning task which requires a large number of supporting facts, which other methods are not able to solve, and can generalize past its training horizon. It can also be practically used on large scale datasets such as Children's Book Test, where it obtains competitive performance, reading the story in a single pass.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Question Answering bAbi EntNet Accuracy (trained on 10k) 99.5% # 3
Accuracy (trained on 1k) 89.1% # 2
Mean Error Rate 9.7% # 8
Procedural Text Understanding ProPara EntNet Henaff et al. (2017) Sentence-level Cat 1 (Accuracy) 51.6 # 7
Sentence-level Cat 2 (Accuracy) 18.8 # 6
Sentence-level Cat 3 (Accuracy) 7.8 # 7
Document level (P) 50.2 # 6
Document level (R) 33.5 # 5
Document level (F1) 40.2 # 5