Deep Learning on Graphs: A Survey

Ziwei Zhang, Peng Cui, Wenwu Zhu

Main figure for Deep Learning on Graphs: A Survey

Abstract

Deep learning has been shown to be successful in a number of domains, ranging from acoustics, images, to natural language processing. However, applying deep learning to the ubiquitous graph data is non-trivial because of the unique characteristics of graphs. Recently, substantial research efforts have been devoted to applying deep learning methods to graphs, resulting in beneficial advances in graph analysis techniques. In this survey, we comprehensively review the different types of deep learning methods on graphs. We divide the existing methods into five categories based on their model architectures and training strategies: graph recurrent neural networks, graph convolutional networks, graph autoencoders, graph reinforcement learning, and graph adversarial methods. We then provide a comprehensive overview of these methods in a systematic manner mainly by following their development history. We also analyze the differences and compositions of different methods. Finally, we briefly outline the applications in which they have been used and discuss potential future research directions.

Benchmark Results

Tested on 5 datasets 2 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

#ModelAccuracy ↑Paper
1
GraphSAGE+CNA
0.9418 ±0.0033
2
GCN+CNA
0.9366 ±0.0048
3
TransformerConv
0.9350 ±0.0058
21
GCN
actual #21 vs paper #573 552 places worse
0.9004
best reported
actual #21 vs paper #573 552 places worse
paper 0.8150
paper 0.8150 Δ at pub 0.0567 Δ today 0.0854
39
SAGE
0.8911 ±0.0012
44
GAT
actual #44 vs paper #424 380 places worse
0.8897 ±0.0014
best reported
actual #44 vs paper #424 380 places worse
paper 0.8300 ±0.0070
95
RGCN
actual #95 vs paper #439 344 places worse
0.8801 ±0.0047
best reported
actual #95 vs paper #439 344 places worse
paper 0.8280 ±0.0060
139
ChebNet
actual #139 vs paper #589 450 places worse
0.8712 ±0.0060
best reported
actual #139 vs paper #589 450 places worse
paper 0.8120
paper 0.8120 Δ at pub 0.0592
177
DGI
actual #177 vs paper #486 309 places worse
0.8630 ±0.0020
best reported
actual #177 vs paper #486 309 places worse
paper 0.8230 ±0.0060
paper 0.8230 ±0.0060 Δ today 0.0400
223
SGC
actual #223 vs paper #606 383 places worse
0.8509 ±0.0648
best reported
actual #223 vs paper #606 383 places worse
paper 0.8100

Difference within reported uncertainty; not counted as underperformance.

paper 0.8100 Δ today 0.0409
226
FastGCN
actual #226 vs paper #803 577 places worse
0.8500
best reported
actual #226 vs paper #803 577 places worse
paper 0.7230
368
DGCN
0.8350
This paper
391
LGCN
0.8330 ±0.0050
This paper
426
GraphSGAN
0.8300 ±0.0130
This paper
442
GWNN
0.8280
This paper
508
MoNet
actual #508 vs paper #558 50 places worse
0.8220 ±0.0070
best reported
actual #508 vs paper #558 50 places worse
paper 0.8170 ±0.0050
best at pub: Graph Attention Networks (ICLR 2017)
best today: GRAND: Graph Neural Diffusion (ICML 2021)

Difference within reported uncertainty; not counted as underperformance.

paper 0.8170 ±0.0050 Δ today 0.0050
531
StochasticGCN
0.8200 ±0.0080
This paper
538
CayleyNets
0.8190 ±0.0070
This paper
724
MLP
0.7768
795
Bruna et al.
0.7330
This paper
#ModelAccuracy ↑Paper
1
CNA
0.9575 ±0.0058
2
IS-GIB
0.9390 ±0.0187
3
EERM
0.9112 ±0.0145
35
GAT
actual #35 vs paper #388 353 places worse
0.8080 ±0.0026
best reported
actual #35 vs paper #388 353 places worse
paper 0.7250 ±0.0070
paper 0.7250 ±0.0070 Δ today 0.0830
44
GCN
actual #44 vs paper #603 559 places worse
0.8033 ±0.0088
best reported
actual #44 vs paper #603 559 places worse
paper 0.7030
61
GraphSAGE
0.7923 ±0.0053
66
DGI
actual #66 vs paper #474 408 places worse
0.7890 ±0.0020
best reported
actual #66 vs paper #474 408 places worse
paper 0.7180 ±0.0070
paper 0.7180 ±0.0070 Δ today 0.0710
96
SGC
actual #96 vs paper #463 367 places worse
0.7765 ±0.0012
best reported
actual #96 vs paper #463 367 places worse
paper 0.7190 ±0.0010
232
ChebNet
actual #232 vs paper #630 398 places worse
0.7399 ±0.0160
best reported
actual #232 vs paper #630 398 places worse
paper 0.6980
paper 0.6980 Δ today 0.0419
317
GraphSGAN
0.7310 ±0.0180
This paper
327
LGCN
0.7300 ±0.0060
This paper
350
MLP
0.7289 ±0.0010
375
DGCN
0.7260
This paper
482
GWNN
0.7170
This paper
502
RGCN
actual #502 vs paper #538 36 places worse
0.7158 ±0.0076
best reported
actual #502 vs paper #538 36 places worse
paper 0.7120 ±0.0050

Difference within reported uncertainty; not counted as underperformance.

paper 0.7120 ±0.0050 Δ today 0.0038
564
StochasticGCN
0.7090 ±0.0020
This paper
837
Bruna
0.5890
This paper
#ModelAccuracy ↑Paper
1
LGTL
0.9518 ±0.0011
2
MixHop (LO)
0.9513 ±0.0038
3
GraphSAGE (LO)
0.9511 ±0.0036
18
GCN
actual #18 vs paper #585 567 places worse
0.9296 ±0.0015
best reported
actual #18 vs paper #585 567 places worse
paper 0.7900
20
GAT
actual #20 vs paper #585 565 places worse
0.9233 ±0.0018
best reported
actual #20 vs paper #585 565 places worse
paper 0.7900 ±0.0030
116
SAGE
0.8881 ±0.0015
141
SGC
actual #141 vs paper #602 461 places worse
0.8832 ±0.0055
best reported
actual #141 vs paper #602 461 places worse
paper 0.7890
159
FastGCN
actual #159 vs paper #823 664 places worse
0.8800
best reported
actual #159 vs paper #823 664 places worse
paper 0.7210
199
ChebNet
actual #199 vs paper #806 607 places worse
0.8718 ±0.0029
best reported
actual #199 vs paper #806 607 places worse
paper 0.7440
paper 0.7440 Δ today 0.1278
204
DGI
actual #204 vs paper #745 541 places worse
0.8708 ±0.0017
best reported
actual #204 vs paper #745 541 places worse
paper 0.7680 ±0.0060
210
MLP
0.8690 ±0.0074
259
RGCN
actual #259 vs paper #574 315 places worse
0.8498 ±0.0016
best reported
actual #259 vs paper #574 315 places worse
paper 0.7910 ±0.0030
paper 0.7910 ±0.0030 Δ today 0.0588
446
MoNet
actual #446 vs paper #614 168 places worse
0.8006 ±0.0055
best reported
actual #446 vs paper #614 168 places worse
paper 0.7880 ±0.0040
450
DGCN
0.8000
This paper
505
LGCN
0.7950 ±0.0020
This paper
576
GWNN
0.7910
This paper
592
StochasticGCN
0.7900 ±0.0040
This paper
813
Bruna et al.
0.7390
This paper
#ModelF1 ↑Paper
1
EnGCN
0.9714 ±0.0003
2
SAGN+SLE
0.9710
3
GAMLP(JK)
0.9704 ±0.0001
14
GraphSAGE
actual #14 vs paper #52 38 places worse
0.9660
best reported
actual #14 vs paper #52 38 places worse
paper 0.9540
21
JK-Nets
0.9650
This paper
30
StochasticGCN
0.9630
This paper
31
Adapt
0.9630 ±0.0030
This paper
58
GCN
0.9530 ±0.0010
73
SGC
0.9490
This paper
82
DGI
actual #82 vs paper #84 2 places worse
0.9414 ±0.0003
best reported
actual #82 vs paper #84 2 places worse
paper 0.9400 ±0.0010

Difference within reported uncertainty; not counted as underperformance.

paper 0.9400 ±0.0010 Δ today 0.0014
91
FastGCN
0.9370
This paper
#ModelF1 ↑Paper
1
GAMLP(JK)
0.9982 ±0.0001
2
GraphCON-GCN
0.9960
3
GCNII
0.9953 ±0.0001
17
GaAN
actual #17 vs paper #17
0.9871
best reported
actual #17 vs paper #17
paper 0.9870
21
GCN
0.9850
39
StochasticGCN
0.9790
This paper
52
JK-Nets
0.9760 ±0.0070
This paper
57
GAT
actual #57 vs paper #57
0.9732 ±0.0032
best reported
actual #57 vs paper #57
paper 0.9730 ±0.0020

Difference within reported uncertainty; not counted as underperformance.

paper 0.9730 ±0.0020 Δ today 0.0002
97
SSE
0.8360
This paper
101
GraphSAGE
actual #101 vs paper #153 52 places worse
0.7860
best reported
actual #101 vs paper #153 52 places worse
paper 0.6120
103
LGCN
0.7720 ±0.0020
This paper
148
DGI
0.6380 ±0.0020
This paper
173
MLP
0.4620
SOTA performance (not significantly different from the best) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best today — OK at publication; today's leaderboard best is elsewhere