Uplifting Message Passing Neural Network with Graph Original Information

Xiao Liu, Lijun Zhang, Hui Guan

Main figure for Uplifting Message Passing Neural Network with Graph Original Information

Abstract

Message passing neural networks (MPNNs) learn the representation of graph-structured data based on graph original information, including node features and graph structures, and have shown astonishing improvement in node classification tasks. However, the expressive power of MPNNs is upper bounded by the first-order Weisfeiler-Leman test and its accuracy still has room for improvement. This work studies how to improve MPNNs’ expressiveness and generalizability by fully exploiting graph original information both theoretically and empirically. It further proposes a new GNN model called INGNN (INformation-enhanced Graph Neural Network) that leverages the insights to improve node classification performance. Extensive experiments on both synthetic and real datasets demonstrate the superiority (av-erage rank 1.78) of our INGNN compared with state-of-the-art methods.

Proposed method: INGNN

Benchmark Results

Tested on 4 datasets 16 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

#ModelAccuracy ↑Paper
1
CNA
0.9575 ±0.0058
2
IS-GIB
0.9390 ±0.0187
3
EERM
0.9112 ±0.0145
35
GAT
actual #35 vs paper #756 721 places worse
0.8080 ±0.0026
best reported
actual #35 vs paper #756 721 places worse
paper 0.6590 ±0.0188
paper 0.6590 ±0.0188 Δ at pub 0.1423 Δ today 0.1490
44
GCN
actual #44 vs paper #802 758 places worse
0.8033 ±0.0088
best reported
actual #44 vs paper #802 758 places worse
paper 0.6336 ±0.0206
paper 0.6336 ±0.0206 Δ at pub 0.1607 Δ today 0.1697
58
BernNet
actual #58 vs paper #859 801 places worse
0.7934 ±0.0032
best reported
actual #58 vs paper #859 801 places worse
paper 0.5352 ±0.0470
paper 0.5352 ±0.0470 Δ today 0.2582
61
GraphSAGE
0.7923 ±0.0053
75
DAGNN
actual #75 vs paper #718 643 places worse
0.7860
best reported
actual #75 vs paper #718 643 places worse
paper 0.6712 ±0.0171
paper 0.6712 ±0.0171 Δ at pub 0.1148
102
GloGNN
actual #102 vs paper #854 752 places worse
0.7740 ±0.0170
best reported
actual #102 vs paper #854 752 places worse
paper 0.5572 ±0.0309
paper 0.5572 ±0.0309 Δ today 0.2168
111
H2GCN
actual #111 vs paper #787 676 places worse
0.7711 ±0.0157
best reported
actual #111 vs paper #787 676 places worse
paper 0.6440 ±0.0144
126
MixHop
actual #126 vs paper #848 722 places worse
0.7626 ±0.0295
best reported
actual #126 vs paper #848 722 places worse
paper 0.5698 ±0.0480
176
GPR-GNN
actual #176 vs paper #781 605 places worse
0.7512 ±0.0198
best reported
actual #176 vs paper #781 605 places worse
paper 0.6472 ±0.0159
paper 0.6472 ±0.0159 Δ at pub 0.1040
312
LINKX
actual #312 vs paper #858 546 places worse
0.7319 ±0.0099
best reported
actual #312 vs paper #858 546 places worse
paper 0.5366 ±0.0369
350
MLP
actual #350 vs paper #864 514 places worse
0.7289 ±0.0010
best reported
actual #350 vs paper #864 514 places worse
paper 0.5094 ±0.0420
paper 0.5094 ±0.0420 Δ at pub 0.1864 Δ today 0.2195
579
ACM-GCN
actual #579 vs paper #770 191 places worse
0.7070 ±0.0081
best reported
actual #579 vs paper #770 191 places worse
paper 0.6508 ±0.0191
paper 0.6508 ±0.0191 Δ today 0.0562
630
INGNN
0.6982 ±0.0108
This paper
801
AdaGNN
0.6344 ±0.0194
This paper
#ModelAccuracy ↑Paper
1
LGTL
0.9518 ±0.0011
2
MixHop (LO)
0.9513 ±0.0038
3
GraphSAGE (LO)
0.9511 ±0.0036
16
H2GCN
actual #16 vs paper #761 745 places worse
0.9361 ±0.0013
best reported
actual #16 vs paper #761 745 places worse
paper 0.7630 ±0.0280
paper 0.7630 ±0.0280 Δ at pub 0.1319 Δ today 0.1731
18
GCN
actual #18 vs paper #665 647 places worse
0.9296 ±0.0015
best reported
actual #18 vs paper #665 647 places worse
paper 0.7812 ±0.0160
20
GAT
actual #20 vs paper #747 727 places worse
0.9233 ±0.0018
best reported
actual #20 vs paper #747 727 places worse
paper 0.7678 ±0.0238
41
ACM-GCN
actual #41 vs paper #750 709 places worse
0.9096 ±0.0062
best reported
actual #41 vs paper #750 709 places worse
paper 0.7670 ±0.0132
paper 0.7670 ±0.0132 Δ today 0.1426
70
MixHop
actual #70 vs paper #765 695 places worse
0.9004 ±0.0141
best reported
actual #70 vs paper #765 695 places worse
paper 0.7614 ±0.0237
paper 0.7614 ±0.0237 Δ at pub 0.0466 Δ today 0.1390
85
GloGNN
actual #85 vs paper #819 734 places worse
0.8960 ±0.0040
best reported
actual #85 vs paper #819 734 places worse
paper 0.7272 ±0.0116
paper 0.7272 ±0.0116 Δ today 0.1688
93
BernNet
actual #93 vs paper #792 699 places worse
0.8933 ±0.0007
best reported
actual #93 vs paper #792 699 places worse
paper 0.7532 ±0.0155
paper 0.7532 ±0.0155 Δ today 0.1401
116
SAGE
0.8881 ±0.0015
162
LINKX
actual #162 vs paper #853 691 places worse
0.8786 ±0.0077
best reported
actual #162 vs paper #853 691 places worse
paper 0.6766 ±0.0429
191
GPR-GNN
actual #191 vs paper #573 382 places worse
0.8738 ±0.0063
best reported
actual #191 vs paper #573 382 places worse
paper 0.7912 ±0.0087
paper 0.7912 ±0.0087 Δ at pub 0.0826
210
MLP
actual #210 vs paper #859 649 places worse
0.8690 ±0.0074
best reported
actual #210 vs paper #859 649 places worse
paper 0.6604 ±0.0229
paper 0.6604 ±0.0229 Δ at pub 0.2034 Δ today 0.2086
226
DAGNN
actual #226 vs paper #658 432 places worse
0.8640
best reported
actual #226 vs paper #658 432 places worse
paper 0.7828 ±0.0158
paper 0.7828 ±0.0158 Δ at pub 0.0812
458
INGNN
0.7998 ±0.0157
This paper
747
AdaGNN
0.7680 ±0.0145
This paper

LINKX Benchmarks

#ModelAccuracy ↑Paper
1
DC-GNN
0.8669 ±0.0022
2
M2M-GNN
0.8594 ±0.0040
3
GloGNN
actual #3 vs paper #3
0.8560 ±0.0027
actual #3 vs paper #3
paper 0.8560 ±0.0027

Improvement within reported uncertainty; not highlighted.

paper 0.8560 ±0.0027 Δ at pub 0.0003
4
INGNN
0.8544 ±0.0060
This paper
11
LINKX
0.8471 ±0.0052
This paper
17
MixHop
0.8363 ±0.0054
This paper
19
AdaGNN
0.8355 ±0.0031
This paper
20
GPR-GNN
actual #20 vs paper #38 18 places worse
0.8354 ±0.0032
best reported
actual #20 vs paper #38 18 places worse
paper 0.8112 ±0.0063
paper 0.8112 ±0.0063 Δ at pub 0.0242 Δ today 0.0242
23
BernNet
actual #23 vs paper #27 4 places worse
0.8326 ±0.0029
best reported
actual #23 vs paper #27 4 places worse
paper 0.8281 ±0.0051

Difference within reported uncertainty; not counted as underperformance.

paper 0.8281 ±0.0051 Δ at pub 0.0045 Δ today 0.0045
24
H2GCN
actual #24 vs paper #35 11 places worse
0.8320 ±0.0055
best reported
actual #24 vs paper #35 11 places worse
paper 0.8154 ±0.0056
paper 0.8154 ±0.0056 Δ today 0.0166
26
ACM-GCN
actual #26 vs paper #28 2 places worse
0.8298 ±0.0063
best reported
actual #26 vs paper #28 2 places worse
paper 0.8268 ±0.0060

Difference within reported uncertainty; not counted as underperformance.

paper 0.8268 ±0.0060 Δ today 0.0030
29
GIN
0.8268 ±0.0032
31
GCN
0.8247 ±0.0027
This paper
36
GAT
actual #36 vs paper #36
0.8153 ±0.0055
best reported
actual #36 vs paper #36
paper 0.8145 ±0.0055

Difference within reported uncertainty; not counted as underperformance.

paper 0.8145 ±0.0055 Δ today 0.0008
51
MLP
actual #51 vs paper #57 6 places worse
0.7534 ±0.0064
best reported
actual #51 vs paper #57 6 places worse
paper 0.7360 ±0.0040
paper 0.7360 ±0.0040 Δ at pub 0.0001 Δ today 0.0174
52
DAGNN
0.7484 ±0.0052
This paper
#ModelAccuracy ↑Paper
1
LargeScaleNet
0.6582 ±0.0036
2
S² DirGCN
0.6495 ±0.0033
3
FaberNet
0.6462 ±0.0101
12
INGNN
0.5653 ±0.0015
This paper
13
LINKX
0.5600 ±0.0134
This paper
16
GloGNN
actual #16 vs paper #16
0.5480 ±0.0030
best reported
actual #16 vs paper #16
paper 0.5452 ±0.0039

Difference within reported uncertainty; not counted as underperformance.

paper 0.5452 ±0.0039 Δ at pub 0.0028
19
MixHop
actual #19 vs paper #19
0.5181 ±0.0017
best reported
actual #19 vs paper #19
paper 0.5178 ±0.0026

Difference within reported uncertainty; not counted as underperformance.

paper 0.5178 ±0.0026 Δ at pub 0.0003 Δ today 0.0003
21
AdaGNN
0.4949 ±0.0016
This paper
22
GAT
actual #22 vs paper #22
0.4937 ±0.0020
actual #22 vs paper #22
paper 0.4937 ±0.0020
paper 0.4937 ±0.0020 Δ at pub 0.0332
23
H2GCN
actual #23 vs paper #23
0.4910 ±0.0010
best reported
actual #23 vs paper #23
paper 0.4909 ±0.0010

Difference within reported uncertainty; not counted as underperformance.

paper 0.4909 ±0.0010 Δ at pub 0.0001
25
ACM-GCN
actual #25 vs paper #25
0.4841 ±0.0030
actual #25 vs paper #25
paper 0.4841 ±0.0030
paper 0.4841 ±0.0030 Δ at pub 0.0104
29
BernNet
actual #29 vs paper #56 27 places worse
0.4634 ±0.0032
best reported
actual #29 vs paper #56 27 places worse
paper 0.3649 ±0.0018
paper 0.3649 ±0.0018 Δ at pub 0.0985
31
GCN
0.4602 ±0.0026
This paper
32
GPR-GNN
actual #32 vs paper #34 2 places worse
0.4597 ±0.0026
best reported
actual #32 vs paper #34 2 places worse
paper 0.4489 ±0.0020
paper 0.4489 ±0.0020 Δ at pub 0.0108
52
DAGNN
0.3849 ±0.0028
This paper
56
MLP
0.3670 ±0.0021
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best at pub — reported value improved over the best known when published Best today — OK at publication; today's leaderboard best is elsewhere