KG-BERT: BERT for Knowledge Graph Completion

7 Sep 2019  ·  Liang Yao, Chengsheng Mao, Yuan Luo ·

Knowledge graphs are important resources for many artificial intelligence tasks but often suffer from incompleteness. In this work, we propose to use pre-trained language models for knowledge graph completion. We treat triples in knowledge graphs as textual sequences and propose a novel framework named Knowledge Graph Bidirectional Encoder Representations from Transformer (KG-BERT) to model these triples. Our method takes entity and relation descriptions of a triple as input and computes scoring function of the triple with the KG-BERT language model. Experimental results on multiple benchmark knowledge graphs show that our method can achieve state-of-the-art performance in triple classification, link prediction and relation prediction tasks.

PDF Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Link Prediction FB15k-237 KG-BERT Hits@10 0.42 # 65
MR 153 # 8
Link Prediction UMLS KG-BERT Hits@10 0.990 # 5
MR 1.47 # 4
Link Prediction WN18RR KG-BERT Hits@10 0.524 # 60
MR 97 # 7

Methods