Densely Connected Attention Propagation for Reading Comprehension

NeurIPS 2018 Yi Tay • Luu Anh Tuan • Siu Cheung Hui • Jian Su

We propose DecaProp (Densely Connected Attention Propagation), a new densely connected neural architecture for reading comprehension (RC). Firstly, our model densely connects all pairwise layers of the network, modeling relationships between passage and query across all hierarchical levels. Secondly, the dense connectors in our network are learned via attention instead of standard residual skip-connectors.

Full paper

Evaluation


Task Dataset Model Metric name Metric value Global rank Compare
Question Answering NarrativeQA DecaProp BLEU-1 44.35 # 1
Question Answering NarrativeQA DecaProp BLEU-4 27.61 # 1
Question Answering NarrativeQA DecaProp METEOR 21.80 # 1
Question Answering NarrativeQA DecaProp Rouge-L 44.69 # 1
Question Answering NewsQA DecaProp F1 66.3 # 1
Question Answering NewsQA DecaProp EM 53.1 # 1
Open-Domain Question Answering Quasar DecaProp EM (Quasar-T) 38.6 # 2
Open-Domain Question Answering Quasar DecaProp F1 (Quasar-T) 46.9 # 2
Open-Domain Question Answering SearchQA DecaProp Unigram Acc 62.2 # 1
Open-Domain Question Answering SearchQA DecaProp N-gram F1 70.8 # 1
Open-Domain Question Answering SearchQA DecaProp EM 56.8 # 1
Open-Domain Question Answering SearchQA DecaProp F1 63.6 # 1