Position: Don't be Afraid of Over-Smoothing And Over-Squashing

Niklas Kormann, Benjamin Doerr, Johannes F. Lutzeyer

Main figure for Position: Don't be Afraid of Over-Smoothing And Over-Squashing

Abstract

Over-smoothing and over-squashing have been extensively studied in the literature on Graph Neural Networks (GNNs) over the past years. We challenge this prevailing focus in GNN research, arguing that these phenomena are less critical for practical applications than assumed. We suggest that performance decreases often stem from uninformative receptive fields rather than over-smoothing. We support this position with extensive experiments on several standard benchmark datasets, demonstrating that accuracy and over-smoothing are mostly uncorrelated and that optimal model depths remain small even with mitigation techniques, thus highlighting the negligible role of over-smoothing. Similarly, we challenge that over-squashing is always detrimental in practical applications. Instead, we posit that the distribution of relevant information over the graph frequently factorises and is often localised within a small k-hop neighbourhood, questioning the necessity of jointly observing entire receptive fields or engaging in an extensive search for long-range interactions. The results of our experiments show that architectural interventions designed to mitigate over-squashing fail to yield significant performance gains. This position paper advocates for a paradigm shift in theoretical research, urging a diligent analysis of learning tasks and datasets using statistics that measure the underlying distribution of label-relevant information to better understand their localisation and factorisation.

Benchmark Results

Tested on 8 datasets 10 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

#ModelAccuracy ↑Paper
1
GraphSAGE+CNA
0.9418 ±0.0033
2
GCN+CNA
0.9366 ±0.0048
3
TransformerConv
0.9350 ±0.0058
21
GCN
actual #21 vs paper #738 717 places worse
0.9004
best reported
actual #21 vs paper #738 717 places worse
paper 0.7710 ±0.0330
paper 0.7710 ±0.0330 Δ at pub 0.1294
39
SAGE
0.8911 ±0.0012
45
GCNII
actual #45 vs paper #756 711 places worse
0.8893 ±0.0137
best reported
actual #45 vs paper #756 711 places worse
paper 0.7630 ±0.0230
best at pub: Language is All a Graph Needs (2023)
paper 0.7630 ±0.0230 Δ at pub 0.1263 Δ today 0.1263
723
EGP (GCN)
0.7770 ±0.0470
This paper
724
MLP
0.7768
758
GCN + SDRF
0.7620 ±0.0300
This paper
765
G²
0.7560 ±0.0110
This paper
769
GCN + BORF
0.7540 ±0.0030
This paper
820
GCN + Pairnorm
0.6960 ±0.0550
This paper
#ModelAccuracy ↑Paper
1
CNA
0.9575 ±0.0058
2
IS-GIB
0.9390 ±0.0187
3
EERM
0.9112 ±0.0145
36
GCNII
actual #36 vs paper #98 62 places worse
0.8076 ±0.0030
best reported
actual #36 vs paper #98 62 places worse
paper 0.7760 ±0.0020
paper 0.7760 ±0.0020 Δ at pub 0.0316
44
GCN
actual #44 vs paper #155 111 places worse
0.8033 ±0.0088
best reported
actual #44 vs paper #155 111 places worse
paper 0.7550 ±0.0080
paper 0.7550 ±0.0080 Δ at pub 0.0483
61
GraphSAGE
0.7923 ±0.0053
124
G²
0.7650 ±0.0040
This paper
133
GCN + SDRF
0.7610 ±0.0060
This paper
142
GCN + BORF
0.7590 ±0.0050
This paper
150
EGP (GCN)
0.7570 ±0.0060
This paper
350
MLP
0.7289 ±0.0010
678
GCN + Pairnorm
0.6830 ±0.0160
This paper
#ModelAccuracy ↑Paper
1
LGTL
0.9518 ±0.0011
2
MixHop (LO)
0.9513 ±0.0038
3
GraphSAGE (LO)
0.9511 ±0.0036
18
GCN
actual #18 vs paper #172 154 places worse
0.9296 ±0.0015
best reported
actual #18 vs paper #172 154 places worse
paper 0.8770 ±0.0040
58
GCNII
actual #58 vs paper #125 67 places worse
0.9030 ±0.0043
best reported
actual #58 vs paper #125 67 places worse
paper 0.8860 ±0.0020
116
SAGE
0.8881 ±0.0015
161
GCN + Pairnorm
0.8790 ±0.0040
This paper
174
GCN + BORF
0.8770 ±0.0040
This paper
175
GCN + SDRF
0.8770 ±0.0030
This paper
182
G²
0.8760 ±0.0110
This paper
202
EGP (GCN)
0.8710 ±0.0040
This paper
210
MLP
0.8690 ±0.0074

TU Dortmund

#ModelAccuracy ↑Paper
1
MSH-GNN
0.9910 ±0.0030
2
ID-GNN+BEC
0.9830 ±0.0360
3
SupCosine
0.9830 ±0.0250
39
GIN
actual #39 vs paper #325 286 places worse
0.9330 ±0.0290
best reported
actual #39 vs paper #325 286 places worse
paper 0.8820 ±0.0370

Difference within reported uncertainty; not counted as underperformance.

paper 0.8820 ±0.0370 Δ at pub 0.0510
63
GCN
0.9220 ±0.0440
111
EGP (GIN)
0.9110 ±0.0140
This paper
169
GATv2
actual #169 vs paper #169
0.9040 ±0.0430
actual #169 vs paper #169
paper 0.9040 ±0.0430
paper 0.9040 ±0.0430 Δ at pub 0.0690
158
GraphSAGE
actual #158 vs paper #280 122 places worse
0.9040 ±0.0780
best reported
actual #158 vs paper #280 122 places worse
paper 0.8890 ±0.0150

Difference within reported uncertainty; not counted as underperformance.

paper 0.8890 ±0.0150 Δ at pub 0.0150
234
GIN + SDRF
0.8950 ±0.0260
This paper
328
GIN + BORF
0.8810 ±0.0270
This paper
329
GraphSAGE + SDRF
0.8810 ±0.0300
This paper
386
GATv2 + SDRF
0.8720 ±0.0270
This paper
403
MLP
0.8660 ±0.0495
436
GraphSAGE + BORF
0.8560 ±0.0240
This paper
534
GATv2 + BORF
0.8280 ±0.0330
This paper
#ModelAccuracy ↑Paper
1
GMN
0.7866
2
MemGNN
0.7550
3
TFGW SP
0.7510 ±0.0500
17
GatedGCN
0.7000 ±0.0494
26
GraphSAGE
0.6817 ±0.0545
32
GIN
actual #32 vs paper #89 57 places worse
0.6767 ±0.0583
best reported
actual #32 vs paper #89 57 places worse
paper 0.6060 ±0.0340
best at pub: GNNBenchmark (JMLR 2023)
paper 0.6060 ±0.0340 Δ at pub 0.0707
110
GIN + BORF
0.5840 ±0.0210
This paper
124
MLP
0.5583 ±0.0352
157
GIN + SDRF
0.5030 ±0.0260
This paper
159
EGP (GIN)
0.5020 ±0.0270
This paper
166
GATv2
actual #166 vs paper #198 32 places worse
0.4770 ±0.0210
best reported
actual #166 vs paper #198 32 places worse
paper 0.3770 ±0.0280
paper 0.3770 ±0.0280 Δ at pub 0.1000
193
GraphSAGE + SDRF
0.3870 ±0.0220
This paper
204
GATv2 + BORF
0.3740 ±0.0410
This paper
210
GATv2 + SDRF
0.3530 ±0.0310
This paper
226
GraphSAGE + BORF
0.3100 ±0.0250
This paper

OGB

#ModelAccuracy ↑Paper
1
GIANT-XRT
0.7612 ±0.0016
2
GEGCN
0.7576 ±0.0064
3
AGDN
0.7522 ±0.0007
23
GCN
actual #23 vs paper #108 85 places worse
0.7300 ±0.0017
best reported
actual #23 vs paper #108 85 places worse
paper 0.5570 ±0.0020
25
GraphSAGE
0.7277 ±0.0016
107
MLP
0.5765 ±0.0012
108
EGP (GCN)
0.5760 ±0.0030
This paper

LRGB

#ModelAP ↑Paper
1
S^2GCN
0.7311 ±0.0066
2
S4G
0.7293 ±0.0004
3
BuNN
0.7276 ±0.0065
4
GCN
0.7261 ±0.0011
26
GIN
actual #26 vs paper #134 108 places worse
0.6070 ±0.0100
actual #26 vs paper #134 108 places worse
paper 0.6070 ±0.0100
paper 0.6070 ±0.0100 Δ at pub 0.0572 Δ today 0.0989
140
GIN + BORF
0.5970 ±0.0060
This paper
161
GIN + SDRF
0.5520 ±0.0080
This paper
180
EGP (GIN)
0.4290 ±0.0060
This paper
181
MLP
0.4060 ±0.0021

Heterophily Benchmark

#ModelAccuracy ↑Paper
1
Dir-Poly
0.9451 ±0.0022
2
D-CDE
0.9387 ±0.0041
3
LargeScaleNet
0.9358 ±0.0024
26
GraphSAGE
0.9106 ±0.0027
77
GCN
actual #77 vs paper #198 121 places worse
0.8272 ±0.0082
best reported
actual #77 vs paper #198 121 places worse
paper 0.4150 ±0.0070
paper 0.4150 ±0.0070 Δ at pub 0.4122
135
MLP
0.6665 ±0.0047
143
GCNII
0.6510 ±0.0060
This paper
158
G²-GCN
0.6290 ±0.0290
This paper
197
EGP (GCN)
0.4190 ±0.0020
This paper
198
GCN + SDRF
0.4160 ±0.0070
This paper
199
GCN + BORF
0.4090 ±0.0070
This paper
203
GCN + Pairnorm
0.3730 ±0.0040
This paper
SOTA performance (not significantly different from the best) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best at pub — reported value improved over the best known when published Improved at pub — later paper surpassed this result