Multi-Perspective Context Matching for Machine Comprehension

13 Dec 2016  ·  Zhiguo Wang, Haitao Mi, Wael Hamza, Radu Florian ·

Previous machine comprehension (MC) datasets are either too small to train end-to-end deep learning models, or not difficult enough to evaluate the ability of current MC techniques. The newly released SQuAD dataset alleviates these limitations, and gives us a chance to develop more realistic MC models. Based on this dataset, we propose a Multi-Perspective Context Matching (MPCM) model, which is an end-to-end system that directly predicts the answer beginning and ending points in a passage. Our model first adjusts each word-embedding vector in the passage by multiplying a relevancy weight computed against the question. Then, we encode the question and weighted passage by using bi-directional LSTMs. For each point in the passage, our model matches the context of this point against the encoded question from multiple perspectives and produces a matching vector. Given those matched vectors, we employ another bi-directional LSTM to aggregate all the information and predict the beginning and ending points. Experimental result on the test set of SQuAD shows that our model achieves a competitive result on the leaderboard.

PDF Abstract

Datasets


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Open-Domain Question Answering SQuAD1.1 MPCM EM 65.5 # 3
Question Answering SQuAD1.1 Multi-Perspective Matching (single model) EM 70.387 # 157
F1 78.784 # 158
Hardware Burden None # 1
Operations per network pass None # 1
Question Answering SQuAD1.1 Multi-Perspective Matching (ensemble) EM 73.765 # 128
F1 81.257 # 137
Hardware Burden None # 1
Operations per network pass None # 1
Question Answering SQuAD1.1 dev MPCM EM 66.1 # 45
F1 75.8 # 47

Methods