Do We Need Anisotropic Graph Neural Networks?

3 Apr 2021  ·  Shyam A. Tailor, Felix L. Opolka, Pietro Liò, Nicholas D. Lane ·

Common wisdom in the graph neural network (GNN) community dictates that anisotropic models -- in which messages sent between nodes are a function of both the source and target node -- are required to achieve state-of-the-art performance. Benchmarks to date have demonstrated that these models perform better than comparable isotropic models -- where messages are a function of the source node only. In this work we provide empirical evidence challenging this narrative: we propose an isotropic GNN, which we call Efficient Graph Convolution (EGC), that consistently outperforms comparable anisotropic models, including the popular GAT or PNA architectures by using spatially-varying adaptive filters. In addition to raising important questions for the GNN community, our work has significant real-world implications for efficiency. EGC achieves higher model accuracy, with lower memory consumption and latency, along with characteristics suited to accelerator implementation, while being a drop-in replacement for existing architectures. As an isotropic model, it requires memory proportional to the number of vertices in the graph ($\mathcal{O}(V)$); in contrast, anisotropic models require memory proportional to the number of edges ($\mathcal{O}(E)$). We demonstrate that EGC outperforms existing approaches across 6 large and diverse benchmark datasets, and conclude by discussing questions that our work raise for the community going forward. Code and pretrained models for our experiments are provided at https://github.com/shyam196/egc.

PDF Abstract

Datasets


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Graph Property Prediction ogbg-code2 MPNN-Max (No Edge Features) Test F1 score 0.1552 ± 0.0022 # 12
Validation F1 score 0.1441 ± 0.0016 # 11
Number of params 10971506 # 12
Ext. data No # 1
Graph Property Prediction ogbg-code2 PNA (No Edge Features) Test F1 score 0.1570 ± 0.0032 # 11
Validation F1 score 0.1453 ± 0.0025 # 10
Number of params 10992050 # 10
Ext. data No # 1
Graph Property Prediction ogbg-code2 EGC-M (No Edge Features) Test F1 score 0.1595 ± 0.0019 # 8
Validation F1 score 0.1464 ± 0.0021 # 8
Number of params 10986002 # 11
Ext. data No # 1
Graph Property Prediction ogbg-code2 EGC-S (No Edge Features) Test F1 score 0.1528 ± 0.0025 # 13
Validation F1 score 0.1427 ± 0.0020 # 13
Number of params 11156530 # 8
Ext. data No # 1
Graph Property Prediction ogbg-molhiv EGC-S (No Edge Features) Test ROC-AUC 0.7721 ± 0.0110 # 31
Validation ROC-AUC 0.8366 ± 0.0074 # 13
Number of params 317013 # 26
Ext. data No # 1
Graph Property Prediction ogbg-molhiv EGC-M (No Edge Features) Test ROC-AUC 0.7818 ± 0.0153 # 27
Validation ROC-AUC 0.8396 ± 0.0097 # 10
Number of params 317265 # 25
Ext. data No # 1
Node Property Prediction ogbn-arxiv EGC-M (100k) Test Accuracy 0.7196 ± 0.0023 # 50
Validation Accuracy 0.7334 ± 0.0013 # 43
Number of params 99464 # 54
Ext. data No # 1
Node Property Prediction ogbn-arxiv EGC-S (100k) Test Accuracy 0.7219 ± 0.0016 # 43
Validation Accuracy 0.7338 ± 0.0022 # 41
Number of params 100648 # 53
Ext. data No # 1

Methods