A Parallel-Hierarchical Model for Machine Comprehension on Sparse Data

Understanding unstructured text is a major goal within natural language processing. Comprehension tests pose questions based on short text passages to evaluate such understanding. In this work, we investigate machine comprehension on the challenging {\it MCTest} benchmark. Partly because of its limited size, prior work on {\it MCTest} has focused mainly on engineering better features. We tackle the dataset with a neural approach, harnessing simple neural networks arranged in a parallel hierarchy. The parallel hierarchy enables our model to compare the passage, question, and answer from a variety of trainable perspectives, as opposed to using a manually designed, rigid feature set. Perspectives range from the word level to sentence fragments to sequences of sentences; the networks operate only on word-embedding representations of text. When trained with a methodology designed to help cope with limited training data, our Parallel-Hierarchical model sets a new state of the art for {\it MCTest}, outperforming previous feature-engineered approaches slightly and previous neural approaches by a significant margin (over 15\% absolute).

PDF Abstract ACL 2016 PDF ACL 2016 Abstract


Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Question Answering MCTest-160 syntax, frame, coreference, and word embedding features Accuracy 75.27% # 1
Question Answering MCTest-500 Parallel-Hierarchical Accuracy 71% # 1
Question Answering MCTest-500 syntax, frame, coreference, and word embedding features Accuracy 69.94% # 2


No methods listed for this paper. Add relevant methods here