Improving Graph Neural Networks with Simple Architecture Design

17 May 2021  ·  Sunil Kumar Maurya, Xin Liu, Tsuyoshi Murata ·

Graph Neural Networks have emerged as a useful tool to learn on the data by applying additional constraints based on the graph structure. These graphs are often created with assumed intrinsic relations between the entities... In recent years, there have been tremendous improvements in the architecture design, pushing the performance up in various prediction tasks. In general, these neural architectures combine layer depth and node feature aggregation steps. This makes it challenging to analyze the importance of features at various hops and the expressiveness of the neural network layers. As different graph datasets show varying levels of homophily and heterophily in features and class label distribution, it becomes essential to understand which features are important for the prediction tasks without any prior information. In this work, we decouple the node feature aggregation step and depth of graph neural network and introduce several key design strategies for graph neural networks. More specifically, we propose to use softmax as a regularizer and "Soft-Selector" of features aggregated from neighbors at different hop distances; and "Hop-Normalization" over GNN layers. Combining these techniques, we present a simple and shallow model, Feature Selection Graph Neural Network (FSGNN), and show empirically that the proposed model outperforms other state of the art GNN models and achieves up to 64% improvements in accuracy on node classification tasks. Moreover, analyzing the learned soft-selection parameters of the model provides a simple way to study the importance of features in the prediction tasks. Finally, we demonstrate with experiments that the model is scalable for large graphs with millions of nodes and billions of edges. read more

PDF Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Node Classification Actor FSGNN Accuracy 37.63 # 7
Node Classification Chameleon FSGNN Accuracy 78.27 # 1
Node Classification Cornell FSGNN Accuracy 87.84 # 1
Node Property Prediction ogbn-papers100M FSGNN Test Accuracy 0.6807 ± 0.0006 # 2
Validation Accuracy 0.7175 ± 0.0007 # 1
Number of params 16453301 # 2
Node Classification Squirrel FSGNN Accuracy 74.10 # 2
Node Classification Texas FSGNN Accuracy 87.30 # 8
Node Classification Wisconsin FSGNN Accuracy 88.43 # 5