Graph Representation Learning via Hard and Channel-Wise Attention Networks

5 Jul 2019  ·  Hongyang Gao, Shuiwang Ji ·

Attention operators have been widely applied in various fields, including computer vision, natural language processing, and network embedding learning. Attention operators on graph data enables learnable weights when aggregating information from neighboring nodes... However, graph attention operators (GAOs) consume excessive computational resources, preventing their applications on large graphs. In addition, GAOs belong to the family of soft attention, instead of hard attention, which has been shown to yield better performance. In this work, we propose novel hard graph attention operator (hGAO) and channel-wise graph attention operator (cGAO). hGAO uses the hard attention mechanism by attending to only important nodes. Compared to GAO, hGAO improves performance and saves computational cost by only attending to important nodes. To further reduce the requirements on computational resources, we propose the cGAO that performs attention operations along channels. cGAO avoids the dependency on the adjacency matrix, leading to dramatic reductions in computational resource requirements. Experimental results demonstrate that our proposed deep models with the new operators achieve consistently better performance. Comparison results also indicates that hGAO achieves significantly better performance than GAO on both node and graph embedding tasks. Efficiency comparison shows that our cGAO leads to dramatic savings in computational resources, making them applicable to large graphs. read more

PDF Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Graph Classification COLLAB hGANet Accuracy 77.48% # 19
Graph Classification D&D hGANet Accuracy 81.71% # 8
Graph Classification IMDb-M hGANet Accuracy 49.06% # 23
Graph Classification MUTAG hGANet Accuracy 90.00% # 16
Graph Classification PROTEINS cGANet Accuracy 78.23% # 11
Graph Classification PROTEINS hGANet Accuracy 78.65% # 8
Graph Classification PROTEINS GANet Accuracy 77.92% # 12
Graph Classification PTC GANet Accuracy 62.94% # 22
Graph Classification PTC hGANet Accuracy 65.02% # 18
Graph Classification PTC cGANet Accuracy 63.53% # 20

Methods


No methods listed for this paper. Add relevant methods here