Stochastic Answer Networks for Machine Reading Comprehension

ACL 2018 Xiaodong LiuYelong ShenKevin DuhJianfeng Gao

We propose a simple yet robust stochastic answer network (SAN) that simulates multi-step reasoning in machine reading comprehension. Compared to previous work such as ReasoNet which used reinforcement learning to determine the number of steps, the unique feature is the use of a kind of stochastic prediction dropout on the answer module (final layer) of the neural network during the training... (read more)

PDF Abstract

Evaluation Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK COMPARE
Question Answering SQuAD1.1 SAN (ensemble model) EM 79.608 # 57
Question Answering SQuAD1.1 SAN (ensemble model) F1 86.496 # 59
Question Answering SQuAD1.1 SAN (single model) EM 76.828 # 83
Question Answering SQuAD1.1 SAN (single model) F1 84.396 # 84
Question Answering SQuAD1.1 dev SAN (single) EM 76.235 # 14
Question Answering SQuAD1.1 dev SAN (single) F1 84.056 # 17
Question Answering SQuAD2.0 SAN (ensemble model) EM 71.316 # 130
Question Answering SQuAD2.0 SAN (ensemble model) F1 73.704 # 140
Question Answering SQuAD2.0 SAN (single model) EM 68.653 # 137
Question Answering SQuAD2.0 SAN (single model) F1 71.439 # 147