SAFRAN: An interpretable, rule-based link prediction method outperforming embedding models

Neural embedding-based machine learning models have shown promise for predicting novel links in knowledge graphs. Unfortunately, their practical utility is diminished by their lack of interpretability. Recently, the fully interpretable, rule-based algorithm AnyBURL yielded highly competitive results on many general-purpose link prediction benchmarks. However, current approaches for aggregating predictions made by multiple rules are affected by redundancies. We improve upon AnyBURL by introducing the SAFRAN rule application framework, which uses a novel aggregation approach called Non-redundant Noisy-OR that detects and clusters redundant rules prior to aggregation. SAFRAN yields new state-of-the-art results for fully interpretable link prediction on the established general-purpose benchmarks FB15K-237, WN18RR and YAGO3-10. Furthermore, it exceeds the results of multiple established embedding-based algorithms on FB15K-237 and WN18RR and narrows the gap between rule-based and embedding-based algorithms on YAGO3-10.

PDF Abstract AKBC 2021 PDF AKBC 2021 Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Link Prediction FB15k-237 SAFRAN MRR 0.389 # 4
Hits@10 0.537 # 29
Hits@1 0.298 # 5
Link Prediction WN18RR SAFRAN (white box, rule based) MRR 0.502 # 7
Hits@10 0.578 # 25
Hits@1 0.459 # 8
Link Prediction YAGO3-10 SAFRAN (white box, rule based) MRR 0.564 # 8
Hits@10 0.693 # 12
Hits@1 0.492 # 7

Methods