An Open Challenge for Inductive Link Prediction on Knowledge Graphs

3 Mar 2022  ·  Mikhail Galkin, Max Berrendorf, Charles Tapley Hoyt ·

An emerging trend in representation learning over knowledge graphs (KGs) moves beyond transductive link prediction tasks over a fixed set of known entities in favor of inductive tasks that imply training on one graph and performing inference over a new graph with unseen entities. In inductive setups, node features are often not available and training shallow entity embedding matrices is meaningless as they cannot be used at inference time with unseen entities. Despite the growing interest, there are not enough benchmarks for evaluating inductive representation learning methods. In this work, we introduce ILPC 2022, a novel open challenge on KG inductive link prediction. To this end, we constructed two new datasets based on Wikidata with various sizes of training and inference graphs that are much larger than existing inductive benchmarks. We also provide two strong baselines leveraging recently proposed inductive methods. We hope this challenge helps to streamline community efforts in the inductive graph representation learning area. ILPC 2022 follows best practices on evaluation fairness and reproducibility, and is available at https://github.com/pykeen/ilpc2022.

PDF Abstract

Datasets


Introduced in the Paper:

ILPC22-Large ILPC22-Small
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Inductive Link Prediction ILPC22-Large Inductive NodePiece + GNN MRR 0.0705 # 1
Hits@100 0.374 # 1
Hits@10 0.1458 # 1
Hits@5 0.099 # 1
Hits@3 0.073 # 1
Hits@1 0.0319 # 2
AMRI 0.682 # 1
Inductive Link Prediction ILPC22-Large Inductive NodePiece MRR 0.0651 # 2
Hits@100 0.287 # 2
Hits@10 0.1246 # 2
Hits@5 0.0809 # 2
Hits@3 0.0542 # 2
Hits@1 0.0373 # 1
AMRI 0.646 # 2
Inductive Link Prediction ILPC22-Small Inductive NodePiece + GNN MRR 0.1326 # 1
Hits@100 0.4705 # 1
Hits@10 0.2509 # 1
Hits@5 0.1899 # 1
Hits@3 0.1396 # 1
Hits@1 0.0763 # 1
AMRI 0.730 # 1
Inductive Link Prediction ILPC22-Small Inductive NodePiece MRR 0.0381 # 2
Hits@100 0.4678 # 2
Hits@10 0.0917 # 2
Hits@5 0.0500 # 2
Hits@3 0.0219 # 2
Hits@1 0.007 # 2
AMRI 0.666 # 2

Methods


No methods listed for this paper. Add relevant methods here