Graph Models
Graph Transformer
Introduced by Dwivedi et al. in A Generalization of Transformer Networks to GraphsThis is Graph Transformer method, proposed as a generalization of Transformer Neural Network architectures, for arbitrary graphs.
Compared to the original Transformer, the highlights of the presented architecture are:
- The attention mechanism is a function of neighborhood connectivity for each node in the graph.
- The position encoding is represented by Laplacian eigenvectors, which naturally generalize the sinusoidal positional encodings often used in NLP.
- The layer normalization is replaced by a batch normalization layer.
- The architecture is extended to have edge representation, which can be critical to tasks with rich information on the edges, or pairwise interactions (such as bond types in molecules, or relationship type in KGs. etc).
Papers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Node Classification | 33 | 5.99% |
| Graph Learning | 32 | 5.81% |
| Graph Representation Learning | 25 | 4.54% |
| Graph Neural Network | 22 | 3.99% |
| Graph Regression | 18 | 3.27% |
| Graph Classification | 17 | 3.09% |
| Link Prediction | 16 | 2.90% |
| Property Prediction | 12 | 2.18% |
| Graph Generation | 12 | 2.18% |
Laplacian PE
Transformer
