Deep Learning on Graphs: A Survey
Abstract
Deep learning has been shown to be successful in a number of domains, ranging from acoustics, images, to natural language processing. However, applying deep learning to the ubiquitous graph data is non-trivial because of the unique characteristics of graphs. Recently, substantial research efforts have been devoted to applying deep learning methods to graphs, resulting in beneficial advances in graph analysis techniques. In this survey, we comprehensively review the different types of deep learning methods on graphs. We divide the existing methods into five categories based on their model architectures and training strategies: graph recurrent neural networks, graph convolutional networks, graph autoencoders, graph reinforcement learning, and graph adversarial methods. We then provide a comprehensive overview of these methods in a systematic manner mainly by following their development history. We also analyze the differences and compositions of different methods. Finally, we briefly outline the applications in which they have been used and discuss potential future research directions.
Benchmark Results
Tested on 5 datasets 2 baseline values reported too low
Tables show presented results in global context (best 3 papers + GCN + MLP).
Classic
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | GraphSAGE+CNA | 0.9418
±0.0033 | |
| 2 | GCN+CNA | 0.9366
±0.0048 | |
| 3 | TransformerConv | 0.9350
±0.0058 | |
| 21 | GCN | 0.9004 best reported | best at pub: SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels (2017) best today: Understanding Heterophily for Graph Neural Networks (ICML 2024) |
| 39 | SAGE | 0.8911
±0.0012 | |
| 44 | GAT | 0.8897
±0.0014 best reported | best at pub: Topology Adaptive Graph Convolutional Networks (2017) |
| 95 | RGCN | 0.8801
±0.0047 best reported | |
| 139 | ChebNet | 0.8712
±0.0060 best reported | best at pub: SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels (2017) |
| 177 | DGI | 0.8630
±0.0020 best reported | best today: Oversmoothing: A Nightmare for Graph Contrastive Learning? (2023) |
| 223 | SGC | 0.8509
±0.0648 best reported | Difference within reported uncertainty; not counted as underperformance. |
| 226 | FastGCN | 0.8500 best reported | best today: Training Robust Graph Neural Networks with Topology Adaptive Edge Dropping (2021) |
| 368 | DGCN | 0.8350 | This paper |
| 391 | LGCN | 0.8330
±0.0050 | This paper |
| 426 | GraphSGAN | 0.8300
±0.0130 | This paper |
| 442 | GWNN | 0.8280 | This paper |
| 508 | MoNet | 0.8220
±0.0070 best reported | best at pub: Graph Attention Networks (ICLR 2017) best today: GRAND: Graph Neural Diffusion (ICML 2021) Difference within reported uncertainty; not counted as underperformance. |
| 531 | StochasticGCN | 0.8200
±0.0080 | This paper |
| 538 | CayleyNets | 0.8190
±0.0070 | This paper |
| 724 | MLP | 0.7768 | |
| 795 | Bruna et al. | 0.7330 | This paper |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | CNA | 0.9575
±0.0058 | |
| 2 | IS-GIB | 0.9390
±0.0187 | |
| 3 | EERM | 0.9112
±0.0145 | |
| 35 | GAT | 0.8080
±0.0026 best reported | best at pub: Topology Adaptive Graph Convolutional Networks (2017) best today: Feature Expansion for Graph Neural Networks (ICML 2023) |
| 44 | GCN | 0.8033
±0.0088 best reported | best at pub: Topology Adaptive Graph Convolutional Networks (2017) |
| 61 | GraphSAGE | 0.7923
±0.0053 | |
| 66 | DGI | 0.7890
±0.0020 best reported | best today: Oversmoothing: A Nightmare for Graph Contrastive Learning? (2023) |
| 96 | SGC | 0.7765
±0.0012 best reported | |
| 232 | ChebNet | 0.7399
±0.0160 best reported | best at pub: Topology Adaptive Graph Convolutional Networks (2017) best today: Non-backtracking Graph Neural Networks (TMLR 2023) |
| 317 | GraphSGAN | 0.7310
±0.0180 | This paper |
| 327 | LGCN | 0.7300
±0.0060 | This paper |
| 350 | MLP | 0.7289
±0.0010 | |
| 375 | DGCN | 0.7260 | This paper |
| 482 | GWNN | 0.7170 | This paper |
| 502 | RGCN | 0.7158
±0.0076 best reported | best today: Understanding Structural Vulnerability in Graph Convolutional Networks (IJCAI 2021) Difference within reported uncertainty; not counted as underperformance. |
| 564 | StochasticGCN | 0.7090
±0.0020 | This paper |
| 837 | Bruna | 0.5890 | This paper |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | LGTL | 0.9518
±0.0011 | |
| 2 | MixHop (LO) | 0.9513
±0.0038 | |
| 3 | GraphSAGE (LO) | 0.9511
±0.0036 | |
| 18 | GCN | 0.9296
±0.0015 best reported | best at pub: Topology Adaptive Graph Convolutional Networks (2017) |
| 20 | GAT | 0.9233
±0.0018 best reported | best at pub: Topology Adaptive Graph Convolutional Networks (2017) |
| 116 | SAGE | 0.8881
±0.0015 | |
| 141 | SGC | 0.8832
±0.0055 best reported | |
| 159 | FastGCN | 0.8800 best reported | best today: Training Robust Graph Neural Networks with Topology Adaptive Edge Dropping (2021) |
| 199 | ChebNet | 0.8718
±0.0029 best reported | best at pub: Topology Adaptive Graph Convolutional Networks (2017) best today: Non-backtracking Graph Neural Networks (TMLR 2023) |
| 204 | DGI | 0.8708
±0.0017 best reported | |
| 210 | MLP | 0.8690
±0.0074 | |
| 259 | RGCN | 0.8498
±0.0016 best reported | best today: Understanding Structural Vulnerability in Graph Convolutional Networks (IJCAI 2021) |
| 446 | MoNet | 0.8006
±0.0055 best reported | best at pub: Topology Adaptive Graph Convolutional Networks (2017) |
| 450 | DGCN | 0.8000 | This paper |
| 505 | LGCN | 0.7950
±0.0020 | This paper |
| 576 | GWNN | 0.7910 | This paper |
| 592 | StochasticGCN | 0.7900
±0.0040 | This paper |
| 813 | Bruna et al. | 0.7390 | This paper |
| # | Model | F1 ↑ | Paper |
|---|---|---|---|
| 1 | EnGCN | 0.9714
±0.0003 | |
| 2 | SAGN+SLE | 0.9710 | |
| 3 | GAMLP(JK) | 0.9704
±0.0001 | |
| 14 | GraphSAGE | 0.9660 best reported | best at pub: GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs (2018) best today: Accelerating Large Scale Real-Time GNN Inference using Channel Pruning (2021) |
| 21 | JK-Nets | 0.9650 | This paper |
| 30 | StochasticGCN | 0.9630 | This paper |
| 31 | Adapt | 0.9630
±0.0030 | This paper |
| 58 | GCN | 0.9530
±0.0010 | |
| 73 | SGC | 0.9490 | This paper |
| 82 | DGI | 0.9414
±0.0003 best reported | best today: Calibrating and Improving Graph Contrastive Learning (TMLR 2021) Difference within reported uncertainty; not counted as underperformance. |
| 91 | FastGCN | 0.9370 | This paper |
| # | Model | F1 ↑ | Paper |
|---|---|---|---|
| 1 | GAMLP(JK) | 0.9982
±0.0001 | |
| 2 | GraphCON-GCN | 0.9960 | |
| 3 | GCNII | 0.9953
±0.0001 | |
| 17 | GaAN | 0.9871 best reported | |
| 21 | GCN | 0.9850 | |
| 39 | StochasticGCN | 0.9790 | This paper |
| 52 | JK-Nets | 0.9760
±0.0070 | This paper |
| 57 | GAT | 0.9732
±0.0032 best reported | best at pub: GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs (2018) best today: HAT-GAE: Self-Supervised Graph Auto-encoders with Hierarchical Adaptive Masking and Trainable Corruption (2023) Difference within reported uncertainty; not counted as underperformance. |
| 97 | SSE | 0.8360 | This paper |
| 101 | GraphSAGE | 0.7860 best reported | best at pub: GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs (2018) best today: Optimization-Induced Graph Implicit Nonlinear Diffusion (ICML 2022) |
| 103 | LGCN | 0.7720
±0.0020 | This paper |
| 148 | DGI | 0.6380
±0.0020 | This paper |
| 173 | MLP | 0.4620 |
