Relation Prediction as an Auxiliary Training Objective for Improving Multi-Relational Graph Representations

Learning good representations on multi-relational graphs is essential to knowledge base completion (KBC). In this paper, we propose a new self-supervised training objective for multi-relational graph representation learning, via simply incorporating relation prediction into the commonly used 1vsAll objective. The new training objective contains not only terms for predicting the subject and object of a given triple, but also a term for predicting the relation type. We analyse how this new objective impacts multi-relational learning in KBC: experiments on a variety of datasets and models show that relation prediction can significantly improve entity ranking, the most widely used evaluation task for KBC, yielding a 6.1% increase in MRR and 9.9% increase in Hits@1 on FB15k-237 as well as a 3.1% increase in MRR and 3.4% in Hits@1 on Aristo-v4. Moreover, we observe that the proposed objective is especially effective on highly multi-relational datasets, i.e. datasets with a large number of predicates, and generates better representations when larger embedding sizes are used.

PDF Abstract AKBC 2021 PDF AKBC 2021 Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Link Prediction Aristo-v4 ComplEx-N3-RP MRR 0.311 # 1
Hits@1 0.24 # 1
Hits@3 0.336 # 1
Hits@10 0.447 # 1
Link Prediction CoDEx Large ComplEx-N3-RP MRR 0.345 # 1
Hits@1 0.277 # 1
Hits@3 0.377 # 1
Hits@10 0.473 # 1
Link Prediction CoDEx Medium ComplEx-N3-RP MRR 0.352 # 1
Hits@1 0.277 # 1
Hits@3 0.386 # 1
Hits@10 0.490 # 1
Link Prediction CoDEx Small ComplEx-N3-RP MRR 0.473 # 1
Hits@1 0.375 # 1
Hits@3 0.514 # 1
Hits@10 0.663 # 1
Link Prediction FB15k-237 CP-N3-RP MRR 0.366 # 12
Hits@10 0.55 # 11
Link Prediction FB15k-237 ComplEx-N3-RP MRR 0.389 # 4
Hits@10 0.568 # 4
Hits@3 0.424 # 3
Hits@1 0.298 # 5
MR 163 # 13
Link Prediction FB15k-237 TuckER-RP MRR 0.354 # 28
Hits@10 0.535 # 32
Hits@3 0.388 # 24
Hits@1 0.264 # 23
Link Property Prediction ogbl-biokg ComplEx-N3-RP Test MRR 0.8494 # 7
Validation MRR 0.8497 # 7
Number of params 187750000 # 11
Link Property Prediction ogbl-wikikg2 ComplEx-N3-RP (50dim) Validation MRR 0.6594 # 14
Test MRR 0.6364 # 15
Number of params 250167400 # 15
Link Property Prediction ogbl-wikikg2 ComplEx-N3-RP (100dim) Validation MRR 0.6701 # 12
Test MRR 0.6481 # 13
Number of params 500334800 # 18
Link Prediction WN18RR ComplEx-N3-RP MRR 0.488 # 26
Hits@10 0.578 # 26
Hits@3 0.505 # 25
Hits@1 0.443 # 25

Methods