Cognitive Graph for Multi-Hop Reading Comprehension at Scale

We propose a new CogQA framework for multi-hop question answering in web-scale documents. Inspired by the dual process theory in cognitive science, the framework gradually builds a \textit{cognitive graph} in an iterative process by coordinating an implicit extraction module (System 1) and an explicit reasoning module (System 2). While giving accurate answers, our framework further provides explainable reasoning paths. Specifically, our implementation based on BERT and graph neural network efficiently handles millions of documents for multi-hop reasoning questions in the HotpotQA fullwiki dataset, achieving a winning joint $F_1$ score of 34.9 on the leaderboard, compared to 23.6 of the best competitor.

PDF Abstract ACL 2019 PDF ACL 2019 Abstract

Datasets


Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Question Answering HotpotQA Cognitive Graph QA ANS-EM 0.371 # 51
ANS-F1 0.489 # 51
SUP-EM 0.228 # 47
SUP-F1 0.577 # 48
JOINT-EM 0.124 # 48
JOINT-F1 0.349 # 50

Methods