ICLR 2023

Effective Structural Encodings via Local Curvature Profiles

Lukas Fesser Melanie Weber

Main figure for Effective Structural Encodings via Local Curvature Profiles

Abstract

Structural and Positional Encodings can significantly improve the performance of Graph Neural Networks in downstream tasks. Recent literature has begun to systematically investigate differences in the structural properties that these approaches encode, as well as performance trade-offs between them. However, the question of which structural properties yield the most effective encoding remains open. In this paper, we investigate this question from a geometric perspective. We propose a novel structural encoding based on discrete Ricci curvature (Local Curvature Profiles, short LCP) and show that it significantly outperforms existing encoding approaches. We further show that combining local structural encodings, such as LCP, with global positional encodings improves downstream performance, suggesting that they capture complementary geometric information. Finally, we compare different encoding types with (curvature-based) rewiring techniques. Rewiring has recently received a surge of interest due to its ability to improve the performance of Graph Neural Networks by mitigating over-smoothing and over-squashing effects. Our results suggest that utilizing curvature information for structural encodings delivers significantly larger performance increases than rewiring.

Benchmark Results

Tested on 8 datasets

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

#ModelAccuracy ↑Paper
1
GraphSAGE+CNA
0.9418 ±0.0033
2
GCN+CNA
0.9366 ±0.0048
3
TransformerConv
0.9350 ±0.0058
21
GCN
0.9004
48
GCN (LCP)
0.8890 ±0.0100
This paper
198
GAT (LCP)
0.8570 ±0.0110
This paper
412
CurvGN-n
0.8320 ±0.0090
This paper
421
CurvGN-1
0.8310 ±0.0080
This paper
724
MLP
0.7768
#ModelAccuracy ↑Paper
1
CNA
0.9575 ±0.0058
2
IS-GIB
0.9390 ±0.0187
3
EERM
0.9112 ±0.0145
44
GCN
0.8033 ±0.0088
350
MLP
0.7289 ±0.0010
422
CurvGN-n
0.7240 ±0.0090
This paper
491
CurvGN-1
0.7170 ±0.0100
This paper

TU Dortmund

#ModelAccuracy ↑Paper
1
MSH-GNN
0.9910 ±0.0030
2
ID-GNN+BEC
0.9830 ±0.0360
3
SupCosine
0.9830 ±0.0250
39
GIN
0.9330 ±0.0290
403
MLP
0.8660 ±0.0495
566
GAT (LCP)
0.8200 ±0.0190
This paper
#ModelAccuracy ↑Paper
1
MSH-GNN
0.9410 ±0.0330
2
GRDL
0.8260 ±0.0120
3
GMN
0.8225
26
GAT (LCP, RW)
0.7940 ±0.0170
This paper
32
GIN
0.7880 ±0.0410
140
GIN (LCP, RW)
0.7620 ±0.0140
This paper
149
MLP
0.7600 ±0.0320
#ModelAccuracy ↑Paper
1
GMN
0.7866
2
MemGNN
0.7550
3
TFGW SP
0.7510 ±0.0500
17
GatedGCN
0.7000 ±0.0494
26
GraphSAGE
0.6817 ±0.0545
124
MLP
0.5583 ±0.0352
208
GCN (LCP)
0.3540 ±0.0260
This paper
213
GAT (LCP)
0.3450 ±0.0200
This paper
219
GIN (LCP)
0.3270 ±0.0160
This paper

Heterophilic Graphs

#ModelAccuracy ↑Paper
1
P^2GNN
0.9672 ±0.0147
2
DoG
0.9660 ±0.0210
3
GPF-plus
0.9583 ±0.0419
34
MLP
0.9230 ±0.0070
53
GraphSAGE
0.9000 ±0.0365
414
GIN (LA)
0.6780 ±0.0330
This paper
555
GCN (LCP)
0.5680 ±0.0260
This paper
#ModelAccuracy ↑Paper
1
P^2GNN
0.9541 ±0.0272
2
ACM-GCN
0.9480 ±0.0380
3
RF-GCN
0.9429
24
MLP
0.9136 ±0.0070
138
SAGE
0.8269
473
GIN (LA)
0.5130 ±0.0200
This paper
485
GCN (LCP)
0.5040 ±0.0250
This paper
#ModelAccuracy ↑Paper
1
DMD++
0.9190 ±0.0260
2
F-GREAD-BS
0.9059 ±0.0380
3
APPNP
0.9010 ±0.0350
37
MLP
0.8629 ±0.0450
63
GCN
0.8310 ±0.0320
175
GIN (LA)
0.5780 ±0.0210
This paper
215
GCN (LCP)
0.4740 ±0.0140
This paper
SOTA performance (not significantly different from the best) Baseline / comparison (this paper) Global context (other papers)