Distilling Self-Knowledge From Contrastive Links to Classify Graph Nodes Without Passing Messages

16 Jun 2021  ·  Yi Luo, Aiguo Chen, Ke Yan, Ling Tian ·

Nowadays, Graph Neural Networks (GNNs) following the Message Passing paradigm become the dominant way to learn on graphic data. Models in this paradigm have to spend extra space to look up adjacent nodes with adjacency matrices and extra time to aggregate multiple messages from adjacent nodes. To address this issue, we develop a method called LinkDist that distils self-knowledge from connected node pairs into a Multi-Layer Perceptron (MLP) without the need to aggregate messages. Experiment with 8 real-world datasets shows the MLP derived from LinkDist can predict the label of a node without knowing its adjacencies but achieve comparable accuracy against GNNs in the contexts of semi- and full-supervised node classification. Moreover, LinkDist benefits from its Non-Message Passing paradigm that we can also distil self-knowledge from arbitrarily sampled node pairs in a contrastive way to further boost the performance of LinkDist.

PDF Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Node Classification Amazon Computers CoLinkDistMLP Accuracy 88.85% # 5
Node Classification Amazon Computers LinkDistMLP Accuracy 89.44% # 3
Node Classification Amazon Computers CoLinkDist Accuracy 89.42% # 4
Node Classification Amazon Computers LinkDist Accuracy 89.49% # 2
Node Classification Amazon Photo CoLinkDistMLP Accuracy 94.12% # 3
Node Classification Amazon Photo LinkDistMLP Accuracy 93.83% # 4
Node Classification Amazon Photo CoLinkDist Accuracy 94.36% # 2
Node Classification Amazon Photo LinkDist Accuracy 93.75% # 5
Node Classification Citeseer CoLinkDist Accuracy 75.79% # 15
Node Classification Citeseer CoLinkDistMLP Accuracy 75.77% # 16
Node Classification Citeseer LinkDistMLP Accuracy 75.25% # 20
Node Classification Citeseer LinkDist Accuracy 74.72% # 22
Node Classification CiteSeer with Public Split: fixed 20 nodes per class LinkDist Accuracy 70.27% # 24
Node Classification CiteSeer with Public Split: fixed 20 nodes per class CoLinkDistMLP Accuracy 70.96% # 22
Node Classification CiteSeer with Public Split: fixed 20 nodes per class LinkDistMLP Accuracy 70.26% # 25
Node Classification CiteSeer with Public Split: fixed 20 nodes per class CoLinkDist Accuracy 70.79% # 23
Node Classification Coauthor CS CoLinkDistMLP Accuracy 95.74% # 3
Node Classification Coauthor CS LinkDistMLP Accuracy 95.68% # 4
Node Classification Coauthor CS CoLinkDist Accuracy 95.80% # 2
Node Classification Coauthor CS LinkDist Accuracy 95.66% # 5
Node Classification Coauthor Physics LinkDist Accuracy 96.87% # 4
Node Classification Coauthor Physics LinkDistMLP Accuracy 96.91% # 3
Node Classification Coauthor Physics CoLinkDistMLP Accuracy 96.87% # 4
Node Classification Coauthor Physics CoLinkDist Accuracy 97.05% # 2
Node Classification Cora CoLinkDistMLP Accuracy 87.54% # 14
Node Classification Cora CoLinkDist Accuracy 87.89% # 11
Node Classification Cora LinkDist Accuracy 88.24% # 9
Node Classification Cora LinkDistMLP Accuracy 87.58% # 13
Node Classification Cora Full LinkDistMLP Accuracy 69.53% # 4
Node Classification Cora Full CoLinkDistMLP Accuracy 69.83% # 3
Node Classification Cora Full CoLinkDist Accuracy 70.32% # 1
Node Classification Cora Full LinkDist Accuracy 69.87% # 2
Node Classification Cora Full with Public Split CoLinkDist Accuracy 57.05% # 1
Node Classification Cora Full with Public Split LinkDistMLP Accuracy 51.78% # 4
Node Classification Cora Full with Public Split CoLinkDistMLP Accuracy 53.43% # 3
Node Classification Cora Full with Public Split LinkDist Accuracy 55.87% # 2
Node Classification Cora with Public Split: fixed 20 nodes per class LinkDist Accuracy 81.05% # 23
Node Classification Cora with Public Split: fixed 20 nodes per class CoLinkDistMLP Accuracy 81.19% # 22
Node Classification Cora with Public Split: fixed 20 nodes per class CoLinkDist Accuracy 81.39% # 21
Node Classification Cora with Public Split: fixed 20 nodes per class LinkDistMLP Accuracy 80.79% # 24
Node Property Prediction ogbn-arxiv CoLinkDistMLP Test Accuracy 0.5638 ± 0.0016 # 58
Validation Accuracy 0.5807 ± 0.0011 # 55
Number of params 120912 # 45
Ext. data No # 1
Node Property Prediction ogbn-mag CoLinkDistMLP Test Accuracy 0.2761 ± 0.0018 # 20
Validation Accuracy 0.2646 ± 0.0013 # 20
Number of params 278202 # 20
Ext. data No # 1
Node Property Prediction ogbn-products CoLinkDistMLP Test Accuracy 0.6259 ± 0.0010 # 46
Validation Accuracy 0.7721 ± 0.0015 # 42
Number of params 115806 # 37
Ext. data No # 1
Node Classification Pubmed LinkDistMLP Accuracy 88.79% # 14
Node Classification Pubmed CoLinkDistMLP Accuracy 89.53% # 9
Node Classification Pubmed CoLinkDist Accuracy 89.58% # 8
Node Classification Pubmed LinkDist Accuracy 88.86% # 13
Node Classification PubMed with Public Split: fixed 20 nodes per class LinkDistMLP Accuracy 72.41% # 32
Node Classification PubMed with Public Split: fixed 20 nodes per class CoLinkDistMLP Accuracy 75.41% # 30
Node Classification PubMed with Public Split: fixed 20 nodes per class LinkDist Accuracy 74.06% # 31
Node Classification PubMed with Public Split: fixed 20 nodes per class CoLinkDist Accuracy 75.64% # 29

Methods


No methods listed for this paper. Add relevant methods here