Break the Ceiling: Stronger Multi-scale Deep Graph Convolutional Networks

Recently, neural network based approaches have achieved significant improvement for solving large, complex, graph-structured problems. However, their bottlenecks still need to be addressed, and the advantages of multi-scale information and deep architectures have not been sufficiently exploited. In this paper, we theoretically analyze how existing Graph Convolutional Networks (GCNs) have limited expressive power due to the constraint of the activation functions and their architectures. We generalize spectral graph convolution and deep GCN in block Krylov subspace forms and devise two architectures, both with the potential to be scaled deeper but each making use of the multi-scale information in different ways. We further show that the equivalence of these two architectures can be established under certain conditions. On several node classification tasks, with or without the help of validation, the two new architectures achieve better performance compared to many state-of-the-art methods.

PDF Abstract NeurIPS 2019 PDF NeurIPS 2019 Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Node Classification Chameleon (60%/20%/20% random splits) Snowball-3 1:1 Accuracy 65.49 ± 1.64 # 13
Node Classification Chameleon (60%/20%/20% random splits) Snowball-2 1:1 Accuracy 64.99 ± 2.39 # 14
Node Classification on Non-Homophilic (Heterophilic) Graphs Chameleon(60%/20%/20% random splits) Snowball-3 1:1 Accuracy 65.49 ± 1.64 # 13
Node Classification on Non-Homophilic (Heterophilic) Graphs Chameleon(60%/20%/20% random splits) Snowball-2 1:1 Accuracy 64.99 ± 2.39 # 14
Node Classification CiteSeer (0.5%) Snowball (linear) Accuracy 59.41% # 6
Node Classification CiteSeer (0.5%) Truncated Krylov Accuracy 64.64% # 3
Node Classification CiteSeer (0.5%) Snowball (tanh) Accuracy 62.05% # 4
Node Classification CiteSeer (0.5%) Snowball (linear + tanh) Accuracy 61.99% # 5
Node Classification CiteSeer (1%) Snowball (linear + tanh) Accuracy 67.07% # 4
Node Classification CiteSeer (1%) Snowball (tanh) Accuracy 64.23% # 6
Node Classification CiteSeer (1%) Truncated Krylov Accuracy 69.03% # 2
Node Classification CiteSeer (1%) Snowball (linear) Accuracy 65.85% # 5
Node Classification CiteSeer (60%/20%/20% random splits) Snowball-2 1:1 Accuracy 81.53 ± 1.71 # 14
Node Classification CiteSeer (60%/20%/20% random splits) Snowball-3 1:1 Accuracy 80.93 ± 1.32 # 18
Node Classification CiteSeer with Public Split: fixed 20 nodes per class Snowball (linear) Accuracy 72.85% # 21
Node Classification CiteSeer with Public Split: fixed 20 nodes per class Truncated Krylov Accuracy 73.86% # 11
Node Classification CiteSeer with Public Split: fixed 20 nodes per class Snowball (tanh) Accuracy 73.32% # 16
Node Classification Cora (0.5%) Snowball (linear) Accuracy 69.99% # 5
Node Classification Cora (0.5%) Snowball (linear + tanh) Accuracy 67.76% # 6
Node Classification Cora (0.5%) Snowball (tanh) Accuracy 71.36% # 4
Node Classification Cora (0.5%) Truncated Krylov Accuracy 74.89% # 3
Node Classification Cora (1%) Snowball (linear) Accuracy 73.10% # 6
Node Classification Cora (1%) Snowball (tanh) Accuracy 74.78% # 5
Node Classification Cora (1%) Snowball (linear + tanh) Accuracy 74.79% # 4
Node Classification Cora (1%) Truncated Krylov Accuracy 78.15% # 2
Node Classification Cora (3%) Truncated Krylov Accuracy 81.92% # 3
Node Classification Cora (3%) Snowball (linear) Accuracy 80.96% # 4
Node Classification Cora (3%) Snowball (tanh) Accuracy 80.72% # 5
Node Classification Cora (3%) Snowball (linear + tanh) Accuracy 79.52% # 6
Node Classification Cora (60%/20%/20% random splits) Snowball-3 1:1 Accuracy 89.33 ± 1.3 # 6
Node Classification Cora (60%/20%/20% random splits) Snowball-2 1:1 Accuracy 88.64 ± 1.15 # 17
Node Classification Cora with Public Split: fixed 20 nodes per class Snowball (linear) Accuracy 83.26% # 16
Node Classification Cora with Public Split: fixed 20 nodes per class Truncated Krylov Accuracy 83.16% # 19
Node Classification Cora with Public Split: fixed 20 nodes per class Snowball (tanh) Accuracy 83.19% # 18
Node Classification Cornell (60%/20%/20% random splits) Snowball-2 1:1 Accuracy 82.62 ± 2.34 # 24
Node Classification on Non-Homophilic (Heterophilic) Graphs Cornell (60%/20%/20% random splits) Snowball-2 1:1 Accuracy 82.62 ± 2.34 # 24
Node Classification on Non-Homophilic (Heterophilic) Graphs Cornell (60%/20%/20% random splits) Snowball-3 1:1 Accuracy 82.95 ± 2.1 # 23
Node Classification Cornell (60%/20%/20% random splits) Snowball-3 1:1 Accuracy 82.95 ± 2.1 # 23
Node Classification Film (60%/20%/20% random splits) Snowball-2 1:1 Accuracy 35.97 ± 0.66 # 27
Node Classification Film (60%/20%/20% random splits) Snowball-3 1:1 Accuracy 36.00 ± 1.36 # 25
Node Classification PubMed (0.03%) Snowball (linear) Accuracy 68.12% # 3
Node Classification PubMed (0.03%) Truncated Krylov Accuracy 71.11% # 2
Node Classification PubMed (0.03%) Snowball (linear + tanh) Accuracy 61.94% # 6
Node Classification PubMed (0.03%) Snowball (tanh) Accuracy 62.61% # 5
Node Classification PubMed (0.05%) Snowball (linear) Accuracy 70.04% # 3
Node Classification PubMed (0.05%) Truncated Krylov Accuracy 72.57% # 2
Node Classification PubMed (0.05%) Snowball (tanh) Accuracy 68.99% # 6
Node Classification PubMed (0.05%) Snowball (linear + tanh) Accuracy 69.45% # 5
Node Classification PubMed (0.1%) Snowball (tanh) Accuracy 74.40% # 4
Node Classification PubMed (0.1%) Truncated Krylov Accuracy 77.21% # 1
Node Classification PubMed (0.1%) Snowball (linear) Accuracy 73.83% # 5
Node Classification PubMed (0.1%) Snowball (linear + tanh) Accuracy 75.30% # 3
Node Classification PubMed (60%/20%/20% random splits) Snowball-3 1:1 Accuracy 88.8 ± 0.82 # 22
Node Classification PubMed (60%/20%/20% random splits) Snowball-2 1:1 Accuracy 89.04 ± 0.49 # 20
Node Classification PubMed with Public Split: fixed 20 nodes per class Truncated Krylov Accuracy 81.7% # 6
Node Classification PubMed with Public Split: fixed 20 nodes per class Snowball (linear) Accuracy 79.10% # 21
Node Classification PubMed with Public Split: fixed 20 nodes per class Snowball (tanh) Accuracy 79.16% # 20
Node Classification Squirrel (60%/20%/20% random splits) Snowball-3 1:1 Accuracy 48.25 ± 0.94 # 14
Node Classification Squirrel (60%/20%/20% random splits) Snowball-2 1:1 Accuracy 47.88 ± 1.23 # 15
Node Classification Texas (60%/20%/20% random splits) Snowball-3 1:1 Accuracy 83.11 ± 3.2 # 23
Node Classification Texas (60%/20%/20% random splits) Snowball-2 1:1 Accuracy 83.11 ± 3.2 # 23
Node Classification on Non-Homophilic (Heterophilic) Graphs Texas(60%/20%/20% random splits) Snowball-2 1:1 Accuracy 83.11 ± 3.2 # 22
Node Classification on Non-Homophilic (Heterophilic) Graphs Texas(60%/20%/20% random splits) Snowball-3 1:1 Accuracy 83.11 ± 3.2 # 22
Node Classification Wisconsin (60%/20%/20% random splits) Snowball-3 1:1 Accuracy 69.5 ± 5.01 # 31
Node Classification Wisconsin (60%/20%/20% random splits) Snowball-2 1:1 Accuracy 74.88 ± 3.42 # 27
Node Classification on Non-Homophilic (Heterophilic) Graphs Wisconsin(60%/20%/20% random splits) Snowball-2 1:1 Accuracy 74.88 ± 3.42 # 24
Node Classification on Non-Homophilic (Heterophilic) Graphs Wisconsin(60%/20%/20% random splits) Snowball-3 1:1 Accuracy 69.5 ± 5.01 # 28

Methods