Machine Comprehension Using Match-LSTM and Answer Pointer

29 Aug 2016  ·  Shuohang Wang, Jing Jiang ·

Machine comprehension of text is an important problem in natural language processing. A recently released dataset, the Stanford Question Answering Dataset (SQuAD), offers a large number of real questions and their answers created by humans through crowdsourcing. SQuAD provides a challenging testbed for evaluating machine comprehension algorithms, partly because compared with previous datasets, in SQuAD the answers do not come from a small set of candidate answers and they have variable lengths. We propose an end-to-end neural architecture for the task. The architecture is based on match-LSTM, a model we proposed previously for textual entailment, and Pointer Net, a sequence-to-sequence model proposed by Vinyals et al.(2015) to constrain the output tokens to be from the input sequences. We propose two ways of using Pointer Net for our task. Our experiments show that both of our two models substantially outperform the best results obtained by Rajpurkar et al.(2016) using logistic regression and manually crafted features.

PDF Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Question Answering SQuAD1.1 Match-LSTM with Ans-Ptr (Sentence) EM 54.505 # 195
F1 67.748 # 198
Question Answering SQuAD1.1 Match-LSTM with Bi-Ans-Ptr (Boundary) EM 64.744 # 181
F1 73.743 # 187
Question Answering SQuAD1.1 Match-LSTM with Ans-Ptr (Boundary) EM 60.474 # 191
F1 70.695 # 196
Question Answering SQuAD1.1 Match-LSTM with Ans-Ptr (Boundary) (ensemble) EM 67.901 # 171
F1 77.022 # 178
Question Answering SQuAD1.1 dev Match-LSTM with Bi-Ans-Ptr (Boundary+Search+b) EM 64.1 # 47
F1 64.7 # 52

Methods