ICML 2021

Elastic Graph Neural Networks

Xiaorui Liu, W. Jin, Yao Ma, Yaxin Li, Hua Liu, Yiqi Wang, Ming Yan, Jiliang Tang

Main figure for Elastic Graph Neural Networks

Abstract

While many existing graph neural networks (GNNs) have been proven to perform $ _2$-based graph smoothing that enforces smoothness globally, in this work we aim to further enhance the local smoothness adaptivity of GNNs via $ _1$-based graph smoothing. As a result, we introduce a family of GNNs (Elastic GNNs) based on $ _1$ and $ _2$-based graph smoothing. In particular, we propose a novel and general message passing scheme into GNNs. This message passing algorithm is not only friendly to back-propagation training but also achieves the desired smoothing properties with a theoretical convergence guarantee. Experiments on semi-supervised learning tasks demonstrate that the proposed Elastic GNNs obtain better adaptivity on benchmark datasets and are significantly robust to graph adversarial attacks. The implementation of Elastic GNNs is available at.

Proposed method: ElasticGNN

Benchmark Results

Tested on 3 datasets 7 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

#ModelAccuracy ↑Paper
1
GraphTARIF
0.9703 ±0.0019
2
GAT
actual #2 vs paper #460 458 places worse
0.9669 ±0.0014
best reported
actual #2 vs paper #460 458 places worse
paper 0.8960 ±0.0160
paper 0.8960 ±0.0160 Δ at pub 0.0296 Δ today 0.0709
3
GAT+T
0.9660 ±0.0007
7
SAGE
0.9643 ±0.0027
8
GCN
actual #8 vs paper #438 430 places worse
0.9632 ±0.0008
best reported
actual #8 vs paper #438 430 places worse
paper 0.9030 ±0.0120
paper 0.9030 ±0.0120 Δ at pub 0.0212 Δ today 0.0602
21
APPNP
actual #21 vs paper #408 387 places worse
0.9573 ±0.0012
best reported
actual #21 vs paper #408 387 places worse
paper 0.9080 ±0.0130
50
SGC
actual #50 vs paper #522 472 places worse
0.9511 ±0.0009
best reported
actual #50 vs paper #522 472 places worse
paper 0.8350 ±0.0290
56
GraphSAGE
actual #56 vs paper #402 346 places worse
0.9090 ±0.0100
actual #56 vs paper #402 346 places worse
paper 0.9090 ±0.0100

Improvement within reported uncertainty; not highlighted.

paper 0.9090 ±0.0100 Δ at pub 0.0080 Δ today 0.0418
119
ChebNet
actual #119 vs paper #434 315 places worse
0.9040 ±0.0100
actual #119 vs paper #434 315 places worse
paper 0.9040 ±0.0100
paper 0.9040 ±0.0100 Δ at pub 0.0790 Δ today 0.0337
389
ElasticGNN
0.9130 ±0.0130
This paper
390
MLP
0.9128 ±0.0063
#ModelAccuracy ↑Paper
1
GraphTARIF
0.9461 ±0.0017
2
GCN
actual #2 vs paper #430 428 places worse
0.9412 ±0.0008
best reported
actual #2 vs paper #430 428 places worse
paper 0.7980 ±0.0160
paper 0.7980 ±0.0160 Δ at pub 0.0805 Δ today 0.1432
3
GAT
actual #3 vs paper #432 429 places worse
0.9398 ±0.0022
best reported
actual #3 vs paper #432 429 places worse
paper 0.7930 ±0.0240
paper 0.7930 ±0.0240 Δ at pub 0.0763 Δ today 0.1468
8
SAGE
0.9351 ±0.0006
46
GraphSAGE
actual #46 vs paper #423 377 places worse
0.8070 ±0.0170
actual #46 vs paper #423 377 places worse
paper 0.8070 ±0.0170

Improvement within reported uncertainty; not highlighted.

paper 0.8070 ±0.0170 Δ at pub 0.0030 Δ today 0.1050
58
APPNP
actual #58 vs paper #428 370 places worse
0.9093 ±0.0018
best reported
actual #58 vs paper #428 370 places worse
paper 0.8010 ±0.0210
82
SGC
actual #82 vs paper #462 380 places worse
0.9022 ±0.0021
best reported
actual #82 vs paper #462 380 places worse
paper 0.7300 ±0.0200
216
ChebNet
actual #216 vs paper #420 204 places worse
0.8100 ±0.0200
actual #216 vs paper #420 204 places worse
paper 0.8100 ±0.0200
paper 0.8100 ±0.0200 Δ at pub 0.0810 Δ today 0.0682
320
MLP
0.8463
424
ElasticGNN
0.8070 ±0.0180
This paper
#ModelAccuracy ↑Paper
1
GrokFormer
0.9831 ±0.0018
2
PolyFormer
0.9808 ±0.0027
3
SGFormer
0.9796 ±0.0081
6
GAT
actual #6 vs paper #221 215 places worse
0.9782 ±0.0028
best reported
actual #6 vs paper #221 215 places worse
paper 0.9330 ±0.0070
paper 0.9330 ±0.0070 Δ at pub 0.0290 Δ today 0.0452
9
GCN
actual #9 vs paper #221 212 places worse
0.9774 ±0.0035
best reported
actual #9 vs paper #221 212 places worse
paper 0.9330 ±0.0080
paper 0.9330 ±0.0080 Δ at pub 0.0320 Δ today 0.0444
17
SAGE
0.9739 ±0.0018
42
APPNP
actual #42 vs paper #206 164 places worse
0.9654 ±0.0007
best reported
actual #42 vs paper #206 164 places worse
paper 0.9370 ±0.0070
paper 0.9370 ±0.0070 Δ today 0.0284
43
GraphSAGE
actual #43 vs paper #252 209 places worse
0.9649 ±0.0006
best reported
actual #43 vs paper #252 209 places worse
paper 0.9250 ±0.0080
58
MLP
0.9597 ±0.0007
70
SGC
actual #70 vs paper #232 162 places worse
0.9580 ±0.0010
best reported
actual #70 vs paper #232 162 places worse
paper 0.9310 ±0.0060
paper 0.9310 ±0.0060 Δ today 0.0270
190
ElasticGNN
0.9420 ±0.0050
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Improved at pub — later paper surpassed this result Best today — OK at publication; today's leaderboard best is elsewhere