Position-aware Attention and Supervised Data Improve Slot Filling

Organized relational knowledge in the form of {``}knowledge graphs{''} is important for many applications. However, the ability to populate knowledge bases with facts automatically extracted from documents has improved frustratingly slowly. This paper simultaneously addresses two issues that have held back prior work. We first propose an effective new model, which combines an LSTM sequence model with a form of entity position-aware attention that is better suited to relation extraction. Then we build TACRED, a large (119,474 examples) supervised relation extraction dataset obtained via crowdsourcing and targeted towards TAC KBP relations. The combination of better supervised data and a more appropriate high-capacity model enables much better relation extraction performance. When the model trained on this new dataset replaces the previous relation extraction component of the best TAC KBP 2015 slot filling system, its F1 score increases markedly from 22.2{\%} to 26.7{\%}.

PDF Abstract

Datasets


Introduced in the Paper:

TACRED

Used in the Paper:

SemEval-2010 Task 8 Re-TACRED

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Relation Extraction Re-TACRED PA-LSTM F1 79.4 # 6
Relation Extraction TACRED PA-LSTM F1 65.1 # 30

Methods