Reinforced Mnemonic Reader for Machine Reading Comprehension

8 May 2017  ·  Minghao Hu, Yuxing Peng, Zhen Huang, Xipeng Qiu, Furu Wei, Ming Zhou ·

In this paper, we introduce the Reinforced Mnemonic Reader for machine reading comprehension tasks, which enhances previous attentive readers in two aspects. First, a reattention mechanism is proposed to refine current attentions by directly accessing to past attentions that are temporally memorized in a multi-round alignment architecture, so as to avoid the problems of attention redundancy and attention deficiency... Second, a new optimization approach, called dynamic-critical reinforcement learning, is introduced to extend the standard supervised method. It always encourages to predict a more acceptable answer so as to address the convergence suppression problem occurred in traditional reinforcement learning algorithms. Extensive experiments on the Stanford Question Answering Dataset (SQuAD) show that our model achieves state-of-the-art results. Meanwhile, our model outperforms previous systems by over 6% in terms of both Exact Match and F1 metrics on two adversarial SQuAD datasets. read more

PDF Abstract

Datasets


Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Question Answering SQuAD1.1 Reinforced Mnemonic Reader (ensemble model) EM 82.283 # 43
F1 88.533 # 46
Question Answering SQuAD1.1 Mnemonic Reader (single model) EM 70.995 # 149
F1 80.146 # 147
Question Answering SQuAD1.1 Mnemonic Reader (ensemble) EM 74.268 # 124
F1 82.371 # 126
Question Answering SQuAD1.1 Reinforced Mnemonic Reader (single model) EM 79.545 # 70
F1 86.654 # 70
Question Answering SQuAD1.1 dev R.M-Reader (single) EM 78.9 # 13
F1 86.3 # 15
Question Answering TriviaQA Mnemonic Reader EM 46.94 # 5
F1 52.85 # 10

Methods


No methods listed for this paper. Add relevant methods here