ICML 2023

Randomized Schur Complement Views for Graph Contrastive Learning

Vignesh Kothapalli

Main figure for Randomized Schur Complement Views for Graph Contrastive Learning

Abstract

We introduce a randomized topological augmentor based on Schur complements for Graph Contrastive Learning (GCL). Given a graph laplacian matrix, the technique generates unbiased approximations of its Schur complements and treats the corresponding graphs as augmented views. We discuss the benefits of our approach, provide theoretical justifications and present connections with graph diffusion. Unlike previous efforts, we study the empirical effectiveness of the augmentor in a controlled fashion by varying the design choices for subsequent GCL phases, such as encoding and contrasting. Extensive experiments on node and graph classification benchmarks demonstrate that our technique consistently outperforms pre-defined and adaptive augmentation approaches to achieve state-of-the-art results.

Benchmark Results

Tested on 6 datasets 4 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

#ModelAccuracy ↑Paper
1
GraphTARIF
0.9703 ±0.0019
2
GAT
0.9669 ±0.0014
3
GAT+T
0.9660 ±0.0007
7
SAGE
0.9643 ±0.0027
273
rLap
0.9259 ±0.0105
This paper
326
NodeDropping
0.9215 ±0.0133
This paper
340
EdgeDroppingPR
0.9201 ±0.0087
This paper
390
MLP
0.9128 ±0.0063
399
PPRDiffusion
0.9116 ±0.0091
This paper
402
MarkovDiffusion
0.9091 ±0.0089
This paper
409
EdgeDropping
0.9080 ±0.0055
This paper
413
EdgeDroppingEVC
0.9074 ±0.0090
This paper
443
DeepWalk + feat.
actual #443 vs paper #443
0.9010 ±0.0010
best reported
actual #443 vs paper #443
paper 0.9005 ±0.0008

Difference within reported uncertainty; not counted as underperformance.

paper 0.9005 ±0.0008 Δ at pub 0.0005
444
DeepWalk
actual #444 vs paper #464 20 places worse
0.9005 ±0.0008
best reported
actual #444 vs paper #464 20 places worse
paper 0.8944 ±0.0011
paper 0.8944 ±0.0011 Δ at pub 0.0061 Δ today 0.0061
456
RandomWalkSubgraph
0.8976 ±0.0126
This paper
484
EdgeDroppingDegree
0.8858 ±0.0113
This paper
493
EdgeAddition
0.8764 ±0.0168
This paper
536
Raw Feat
0.7853
This paper
#ModelAccuracy ↑Paper
1
GrokFormer
0.9831 ±0.0018
2
PolyFormer
0.9808 ±0.0027
3
SGFormer
0.9796 ±0.0081
6
GAT
actual #6 vs paper #122 116 places worse
0.9782 ±0.0028
best reported
actual #6 vs paper #122 116 places worse
paper 0.9547 ±0.0015
paper 0.9547 ±0.0015 Δ at pub 0.0116 Δ today 0.0235
9
GCN
actual #9 vs paper #90 81 places worse
0.9774 ±0.0035
best reported
actual #9 vs paper #90 81 places worse
paper 0.9565 ±0.0016
paper 0.9565 ±0.0016 Δ at pub 0.0085 Δ today 0.0209
17
SAGE
0.9739 ±0.0018
58
MLP
0.9597 ±0.0007
66
rLap
0.9583 ±0.0044
This paper
96
NodeDropping
0.9563 ±0.0032
This paper
102
EdgeDroppingEVC
0.9560 ±0.0037
This paper
113
EdgeAddition
0.9553 ±0.0035
This paper
115
EdgeDroppingPR
0.9551 ±0.0025
This paper
123
EdgeDroppingDegree
0.9547 ±0.0038
This paper
136
EdgeDropping
0.9531 ±0.0043
This paper
146
GAE
actual #146 vs paper #161 15 places worse
0.9520 ±0.0010
best reported
actual #146 vs paper #161 15 places worse
paper 0.9492 ±0.0007
paper 0.9492 ±0.0007 Δ today 0.0028
152
MarkovDiffusion
0.9513 ±0.0043
This paper
155
RandomWalkSubgraph
0.9512 ±0.0033
This paper
157
PPRDiffusion
0.9509 ±0.0034
This paper
163
DeepWalk
actual #163 vs paper #262 99 places worse
0.9490
best reported
actual #163 vs paper #262 99 places worse
paper 0.9177 ±0.0015
paper 0.9177 ±0.0015 Δ at pub 0.0313
162
DeepWalk + feat.
0.9490 ±0.0009
This paper
167
DGI
actual #167 vs paper #174 7 places worse
0.9489 ±0.0009
best reported
actual #167 vs paper #174 7 places worse
paper 0.9451 ±0.0052

Difference within reported uncertainty; not counted as underperformance.

paper 0.9451 ±0.0052 Δ at pub 0.0006 Δ today 0.0038
211
Raw Feat
0.9358
This paper

TU Dortmund

#ModelAccuracy ↑Paper
1
MSH-GNN
0.9910 ±0.0030
2
ID-GNN+BEC
0.9830 ±0.0360
3
SupCosine
0.9830 ±0.0250
39
GIN
0.9330 ±0.0290
362
rLap
0.8750 ±0.0986
This paper
403
MLP
0.8660 ±0.0495
449
EdgeDroppingDegree
0.8500 ±0.1162
This paper
450
PPRDiffusion
0.8500 ±0.0671
This paper
489
RandomWalkSubgraph
0.8400 ±0.0917
This paper
508
EdgeDroppingPR
0.8350 ±0.0838
This paper
520
EdgeDropping
0.8300 ±0.0515
This paper
565
NodeDropping
0.8200 ±0.0603
This paper
591
EdgeAddition
0.8100 ±0.0889
This paper
599
EdgeDroppingEVC
0.8050 ±0.1011
This paper
615
MarkovDiffusion
0.8000 ±0.0671
This paper
#ModelAccuracy ↑Paper
1
WL-MLP
0.9954
2
GIN
0.9934
3
GCN
0.9727
131
GraphSAGE
0.8200 ±0.0100
410
rLap
0.7515 ±0.0357
This paper
416
RandomWalkSubgraph
0.7492 ±0.0288
This paper
417
EdgeDropping
0.7489 ±0.0236
This paper
418
EdgeDroppingPR
0.7460 ±0.0229
This paper
423
EdgeDroppingEVC
0.7450 ±0.0186
This paper
425
EdgeDroppingDegree
0.7440 ±0.0165
This paper
442
NodeDropping
0.7365 ±0.0261
This paper
444
EdgeAddition
0.7336 ±0.0181
This paper
445
PPRDiffusion
0.7336 ±0.0234
This paper
462
MarkovDiffusion
0.7183 ±0.0250
This paper
#ModelAccuracy ↑Paper
1
ID-GNN+BEC
0.8150 ±0.0240
2
GIN+BEC
0.8080 ±0.0330
3
GCN+BEC
0.7990 ±0.0340
4
GCN
0.7980 ±0.0230
10
EdgeDroppingDegree
0.7850 ±0.0632
This paper
12
EdgeDropping
0.7830 ±0.1083
This paper
19
EdgeDroppingPR
0.7680 ±0.0748
This paper
23
EdgeAddition
0.7610 ±0.0563
This paper
27
EdgeDroppingEVC
0.7580 ±0.0626
This paper
32
MarkovDiffusion
0.7530 ±0.0836
This paper
42
rLap
0.7480 ±0.1017
This paper
69
PPRDiffusion
0.7350 ±0.0822
This paper
72
WL
actual #72 vs paper #98 26 places worse
0.7340 ±0.0463
best reported
actual #72 vs paper #98 26 places worse
paper 0.7230 ±0.0344

Difference within reported uncertainty; not counted as underperformance.

paper 0.7230 ±0.0344 Δ today 0.0110
125
graph2vec
0.7110 ±0.0054
This paper
134
NodeDropping
0.7030 ±0.0729
This paper
137
RandomWalkSubgraph
0.7000 ±0.0335
This paper
154
Sub2Vec
actual #154 vs paper #154
0.5530 ±0.0150
best reported
actual #154 vs paper #154
paper 0.5526 ±0.0154

Difference within reported uncertainty; not counted as underperformance.

paper 0.5526 ±0.0154 Δ at pub 0.0004
#ModelAccuracy ↑Paper
1
GraphSNN
0.7790 ±0.0360
2
SIN
0.7560 ±0.0320
3
CIN
0.7560 ±0.0370
5
GIN
0.7510 ±0.0510
8
rLap
0.5947 ±0.0742
This paper
9
EdgeDroppingDegree
0.5907 ±0.0500
This paper
10
EdgeAddition
0.5900 ±0.0958
This paper
11
EdgeDroppingPR
0.5780 ±0.0996
This paper
12
RandomWalkSubgraph
0.5780 ±0.0947
This paper
13
NodeDropping
0.5753 ±0.0850
This paper
14
PPRDiffusion
0.5713 ±0.0772
This paper
15
MarkovDiffusion
0.5680 ±0.0771
This paper
16
EdgeDropping
0.5613 ±0.0831
This paper
17
EdgeDroppingEVC
0.5500 ±0.0701
This paper
SOTA performance (not significantly different from the best) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best today — OK at publication; today's leaderboard best is elsewhere