Using Random Noise Equivariantly to Boost Graph Neural Networks Universally

Xiyuan Wang, Muhan Zhang

Main figure for Using Random Noise Equivariantly to Boost Graph Neural Networks Universally

Abstract

Recent advances in Graph Neural Networks (GNNs) have explored the potential of random noise as an input feature to enhance expressivity across diverse tasks. However, naively incorporating noise can degrade performance, while architectures tailored to exploit noise for specific tasks excel yet lack broad applicability. This paper tackles these issues by laying down a theoretical framework that elucidates the increased sample complexity when introducing random noise into GNNs without careful design. We further propose Equivariant Noise GNN (ENGNN), a novel architecture that harnesses the symmetrical properties of noise to mitigate sample complexity and bolster generalization. Our experiments demonstrate that using noise equivariantly significantly enhances performance on node-level, link-level, subgraph, and graph-level tasks and achieves comparable performance to models designed for specific tasks, thereby offering a general method to boost expressivity across various graph tasks.

Proposed method: ENGNN

Benchmark Results

Tested on 3 datasets 10 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

#ModelMAE ↓Paper
1
TIGT
0.0140 ±0.0010
2
FloydNet
0.0160
3
GSSC
0.0190 ±0.0010
7
CIN
actual #7 vs paper #8 1 place worse
0.0210 ±0.0010
best reported
actual #7 vs paper #8 1 place worse
paper 0.0220 ±0.0020

Difference within reported uncertainty; not counted as underperformance.

paper 0.0220 ±0.0020 Δ at pub 0.0010
10
SSWL+
0.0220 ±0.0020
This paper
21
SUN
0.0240 ±0.0030
This paper
24
DRFWL
0.0250 ±0.0030
This paper
33
ESAN
0.0290 ±0.0030
This paper
31
NGNN
0.0290 ±0.0010
This paper
36
MPNN
actual #36 vs paper #54 18 places worse
0.0300
best reported
actual #36 vs paper #54 18 places worse
paper 0.0460 ±0.0020
best at pub: Hyperbolic Graph Neural Networks (NeurIPS 2019)
paper 0.0460 ±0.0020 Δ at pub 0.0050 Δ today 0.0160
52
ENGNN
0.0440 ±0.0020
This paper
56
NMPNN
0.0510 ±0.0040
This paper
69
GIN
0.0880 ±0.0020
This paper
79
GCN
0.1130 ±0.0020

OGB

#ModelHits@20 ↑Paper
1
ELGNN
0.9777 ±0.0037
2
GIDN
0.9542
3
AGDN
0.9538 ±0.0094
16
BUDDY
0.7851 ±0.0136
This paper
17
ENGNN
0.7761 ±0.0450
This paper
20
SAGE
actual #20 vs paper #37 17 places worse
0.6719 ±0.0118
best reported
actual #20 vs paper #37 17 places worse
paper 0.5390 ±0.0474
paper 0.5390 ±0.0474 Δ at pub 0.1329
24
GCN
actual #24 vs paper #53 29 places worse
0.6476 ±0.0145
best reported
actual #24 vs paper #53 29 places worse
paper 0.3707 ±0.0507
paper 0.3707 ±0.0507 Δ at pub 0.2769
26
Neo-GNN
0.6357 ±0.0352
This paper
38
GraphSAGE
0.5390 ±0.0474
43
SEAL
actual #43 vs paper #59 16 places worse
0.4974 ±0.0239
best reported
actual #43 vs paper #59 16 places worse
paper 0.3056 ±0.0386
paper 0.3056 ±0.0386 Δ at pub 0.1918
47
RA
actual #47 vs paper #62 15 places worse
0.4401
best reported
actual #47 vs paper #62 15 places worse
paper 0.2760
paper 0.2760 Δ at pub 0.1641
50
AA
actual #50 vs paper #75 25 places worse
0.3975
best reported
actual #50 vs paper #75 25 places worse
paper 0.1861
paper 0.1861 Δ at pub 0.2114
51
CN
actual #51 vs paper #78 27 places worse
0.3869
best reported
actual #51 vs paper #78 27 places worse
paper 0.1773
paper 0.1773 Δ at pub 0.2096
66
MPNN
0.2476 ±0.1529
This paper
69
NMPNN
0.2308 ±0.0589
This paper
94
NBFnet
0.0400 ±0.0058
This paper
#ModelHits@100 ↑Paper
1
OGB Leader
0.6524 ±0.0150
2
PRB
0.6347 ±0.0175
3
LPFormer
0.6332 ±0.0063
9
RA
actual #9 vs paper #16 7 places worse
0.5324
best reported
actual #9 vs paper #16 7 places worse
paper 0.4933
paper 0.4933 Δ at pub 0.0391
14
SAGE
actual #14 vs paper #62 48 places worse
0.5013 ±0.0055
best reported
actual #14 vs paper #62 48 places worse
paper 0.1655 ±0.0240
paper 0.1655 ±0.0240 Δ at pub 0.3358
15
BUDDY
0.4985 ±0.0020
This paper
18
Neo-GNN
0.4913 ±0.0060
This paper
19
SEAL
0.4880 ±0.0316
This paper
23
MLP
0.4600
29
ENGNN
0.4497 ±0.0074
This paper
33
NMPNN
0.3990 ±0.0552
This paper
39
AA
0.3245
This paper
45
GCN
actual #45 vs paper #59 14 places worse
0.3084 ±0.0178
best reported
actual #45 vs paper #59 14 places worse
paper 0.1867 ±0.0132
paper 0.1867 ±0.0132 Δ at pub 0.1217
49
CN
0.2765
This paper
63
GraphSAGE
0.1655 ±0.0240
70
MPNN
0.0562 ±0.0252
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green)