Neural Semantic Encoders

EACL 2017 Tsendsuren MunkhdalaiHong Yu

We present a memory augmented neural network for natural language understanding: Neural Semantic Encoders. NSE is equipped with a novel memory update rule and has a variable sized encoding memory that evolves over time and maintains the understanding of input sequences through read}, compose and write operations... (read more)

PDF Abstract

Evaluation Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK COMPARE
Natural Language Inference SNLI 300D NSE encoders % Test Accuracy 84.6 # 37
Natural Language Inference SNLI 300D NSE encoders % Train Accuracy 86.2 # 46
Natural Language Inference SNLI 300D NSE encoders Parameters 3.0m # 1
Natural Language Inference SNLI 300D MMA-NSE encoders with attention % Test Accuracy 85.4 # 34
Natural Language Inference SNLI 300D MMA-NSE encoders with attention % Train Accuracy 86.9 # 43
Natural Language Inference SNLI 300D MMA-NSE encoders with attention Parameters 3.2m # 1
Sentiment Analysis SST-2 Binary classification Neural Semantic Encoder Accuracy 89.7 # 22
Question Answering WikiQA MMA-NSE attention MAP 0.6811 # 11
Question Answering WikiQA MMA-NSE attention MRR 0.6993 # 10
Machine Translation WMT2014 English-German NSE-NSE BLEU score 17.9 # 38