Position-aware Attention and Supervised Data Improve Slot Filling

Organized relational knowledge in the form of {}knowledge graphs{''} is important for many applications. However, the ability to populate knowledge bases with facts automatically extracted from documents has improved frustratingly slowly. This paper simultaneously addresses two issues that have held back prior work. We first propose an effective new model, which combines an LSTM sequence model with a form of entity position-aware attention that is better suited to relation extraction. Then we build TACRED, a large (119,474 examples) supervised relation extraction dataset obtained via crowdsourcing and targeted towards TAC KBP relations. The combination of better supervised data and a more appropriate high-capacity model enables much better relation extraction performance. When the model trained on this new dataset replaces the previous relation extraction component of the best TAC KBP 2015 slot filling system, its F1 score increases markedly from 22.2{\%} to 26.7{\%}.

PDF Abstract

Datasets

Introduced in the Paper:

Used in the Paper:

Results from the Paper Add Remove

Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Relation Extraction Re-TACRED PA-LSTM F1 79.4 # 6
Relation Extraction TACRED PA-LSTM F1 65.1 # 30