Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Prediction

Link prediction is a very fundamental task on graphs. Inspired by traditional path-based methods, in this paper we propose a general and flexible representation learning framework based on paths for link prediction. Specifically, we define the representation of a pair of nodes as the generalized sum of all path representations, with each path representation as the generalized product of the edge representations in the path. Motivated by the Bellman-Ford algorithm for solving the shortest path problem, we show that the proposed path formulation can be efficiently solved by the generalized Bellman-Ford algorithm. To further improve the capacity of the path formulation, we propose the Neural Bellman-Ford Network (NBFNet), a general graph neural network framework that solves the path formulation with learned operators in the generalized Bellman-Ford algorithm. The NBFNet parameterizes the generalized Bellman-Ford algorithm with 3 neural components, namely INDICATOR, MESSAGE and AGGREGATE functions, which corresponds to the boundary condition, multiplication operator, and summation operator respectively. The NBFNet is very general, covers many traditional path-based methods, and can be applied to both homogeneous graphs and multi-relational graphs (e.g., knowledge graphs) in both transductive and inductive settings. Experiments on both homogeneous graphs and knowledge graphs show that the proposed NBFNet outperforms existing methods by a large margin in both transductive and inductive settings, achieving new state-of-the-art results.

PDF Abstract NeurIPS 2021 PDF NeurIPS 2021 Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Link Prediction Citeseer NBFNet AUC 92.3% # 9
AP 93.6% # 8
Link Prediction Cora NBFNet AUC 95.6% # 3
AP 96.2% # 2
Link Prediction FB15k-237 NBFNet MRR 0.415 # 1
Hits@10 0.599 # 1
Hits@3 0.454 # 1
Hits@1 0.321 # 1
MR 114 # 4
Link Property Prediction ogbl-biokg NBFNet Test MRR 0.8317 # 10
Validation MRR 0.8318 # 10
Number of params 734,209 # 1
Ext. data No # 1
Link Prediction Pubmed NBFNet AUC 98.3% # 2
AP 98.2% # 2
Link Prediction WN18RR NBFNet MRR 0.551 # 6
Hits@10 0.666 # 11
Hits@3 0.573 # 6
Hits@1 0.497 # 6
MR 636 # 8
Link Prediction YAGO3-10 NBFNet MRR 0.563 # 9
Hits@10 0.708 # 7
Hits@1 0.480 # 8
Hits@3 0.612 # 4

Methods