Model-Driven Graph Contrastive Learning

Ali Azizpour, Nicolas Zilberstein, Santiago Segarra

Main figure for Model-Driven Graph Contrastive Learning

Abstract

We propose MGCL, a model-driven graph contrastive learning (GCL) framework that leverages graphons (probabilistic generative models for graphs) to guide contrastive learning by accounting for the data's underlying generative process. GCL has emerged as a powerful self-supervised framework for learning expressive node or graph representations without relying on annotated labels, which are often scarce in real-world data. By contrasting augmented views of graph data, GCL has demonstrated strong performance across various downstream tasks, such as node and graph classification. However, existing methods typically rely on manually designed or heuristic augmentation strategies that are not tailored to the underlying data distribution and operate at the individual graph level, ignoring similarities among graphs generated from the same model. Conversely, in our proposed approach, MGCL first estimates the graphon associated with the observed data and then defines a graphon-informed augmentation process, enabling data-adaptive and principled augmentations. Additionally, for graph-level tasks, MGCL clusters the dataset and estimates a graphon per group, enabling contrastive pairs to reflect shared semantics and structure. Extensive experiments on benchmark datasets demonstrate that MGCL achieves state-of-the-art performance, highlighting the advantages of incorporating generative models into GCL.

Proposed method: MGCL

Benchmark Results

Tested on 7 datasets 22 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

#ModelAccuracy ↑Paper
1
GraphTARIF
0.9703 ±0.0019
2
GAT
0.9669 ±0.0014
3
GAT+T
0.9660 ±0.0007
7
SAGE
0.9643 ±0.0027
8
GCN
actual #8 vs paper #292 284 places worse
0.9632 ±0.0008
best reported
actual #8 vs paper #292 284 places worse
paper 0.9242 ±0.0022
172
BGRL
actual #172 vs paper #188 16 places worse
0.9337 ±0.0021
best reported
actual #172 vs paper #188 16 places worse
paper 0.9324 ±0.0029

Difference within reported uncertainty; not counted as underperformance.

paper 0.9324 ±0.0029 Δ at pub 0.0013
191
GCA
actual #191 vs paper #244 53 places worse
0.9320 ±0.0060
best reported
actual #191 vs paper #244 53 places worse
paper 0.9278 ±0.0017

Difference within reported uncertainty; not counted as underperformance.

paper 0.9278 ±0.0017 Δ at pub 0.0042 Δ today 0.0042
203
GRACE
actual #203 vs paper #263 60 places worse
0.9313 ±0.0017
best reported
actual #203 vs paper #263 60 places worse
paper 0.9265 ±0.0028
paper 0.9265 ±0.0028 Δ at pub 0.0048
206
MVGRL
actual #206 vs paper #339 133 places worse
0.9310 ±0.0040
best reported
actual #206 vs paper #339 133 places worse
paper 0.9201 ±0.0013
paper 0.9201 ±0.0013 Δ at pub 0.0109
213
DGI
actual #213 vs paper #375 162 places worse
0.9309 ±0.0008
best reported
actual #213 vs paper #375 162 places worse
paper 0.9149 ±0.0025
235
GBT
actual #235 vs paper #266 31 places worse
0.9282 ±0.0040
best reported
actual #235 vs paper #266 31 places worse
paper 0.9263 ±0.0044

Difference within reported uncertainty; not counted as underperformance.

paper 0.9263 ±0.0044 Δ at pub 0.0019
246
MGCL
0.9278 ±0.0023
This paper
320
GraphCL
actual #320 vs paper #320
0.9222 ±0.0042
actual #320 vs paper #320
paper 0.9222 ±0.0042
paper 0.9222 ±0.0042 Δ at pub 0.1052
390
MLP
0.9128 ±0.0063
457
Node2Vec
actual #457 vs paper #458 1 place worse
0.8970 ±0.0010
best reported
actual #457 vs paper #458 1 place worse
paper 0.8967 ±0.0012

Difference within reported uncertainty; not counted as underperformance.

paper 0.8967 ±0.0012 Δ at pub 0.0003 Δ today 0.0003
531
Raw features
actual #531 vs paper #531
0.7999 ±0.0090
actual #531 vs paper #531
paper 0.7999 ±0.0090
paper 0.7999 ±0.0090 Δ at pub 0.0146
#ModelAccuracy ↑Paper
1
GrokFormer
0.9831 ±0.0018
2
PolyFormer
0.9808 ±0.0027
3
SGFormer
0.9796 ±0.0081
9
GCN
actual #9 vs paper #90 81 places worse
0.9774 ±0.0035
best reported
actual #9 vs paper #90 81 places worse
paper 0.9565 ±0.0016
paper 0.9565 ±0.0016 Δ at pub 0.0209
17
SAGE
0.9739 ±0.0018
58
MLP
0.9597 ±0.0007
63
GCA
actual #63 vs paper #88 25 places worse
0.9590 ±0.0020
best reported
actual #63 vs paper #88 25 places worse
paper 0.9568 ±0.0005

Difference within reported uncertainty; not counted as underperformance.

paper 0.9568 ±0.0005 Δ at pub 0.0022
73
MGCL
0.9580 ±0.0008
This paper
72
GraphCL
0.9580 ±0.0014
This paper
69
BGRL
actual #69 vs paper #95 26 places worse
0.9580 ±0.0040
best reported
actual #69 vs paper #95 26 places worse
paper 0.9563 ±0.0004

Difference within reported uncertainty; not counted as underperformance.

paper 0.9563 ±0.0004 Δ at pub 0.0017
77
GRACE
actual #77 vs paper #140 63 places worse
0.9574 ±0.0006
best reported
actual #77 vs paper #140 63 places worse
paper 0.9526 ±0.0006
paper 0.9526 ±0.0006 Δ at pub 0.0048
125
MVGRL
actual #125 vs paper #134 9 places worse
0.9544 ±0.0027
best reported
actual #125 vs paper #134 9 places worse
paper 0.9533 ±0.0003

Difference within reported uncertainty; not counted as underperformance.

paper 0.9533 ±0.0003 Δ at pub 0.0011
149
GBT
actual #149 vs paper #157 8 places worse
0.9520 ±0.0010
best reported
actual #149 vs paper #157 8 places worse
paper 0.9507 ±0.0017

Difference within reported uncertainty; not counted as underperformance.

paper 0.9507 ±0.0017 Δ at pub 0.0013
167
DGI
actual #167 vs paper #174 7 places worse
0.9489 ±0.0009
best reported
actual #167 vs paper #174 7 places worse
paper 0.9451 ±0.0052

Difference within reported uncertainty; not counted as underperformance.

paper 0.9451 ±0.0052 Δ at pub 0.0038
182
Raw features
actual #182 vs paper #182
0.9432 ±0.0014
actual #182 vs paper #182
paper 0.9432 ±0.0014
paper 0.9432 ±0.0014 Δ at pub 0.0072
268
Node2Vec
actual #268 vs paper #268
0.9120 ±0.0010
best reported
actual #268 vs paper #268
paper 0.9119 ±0.0004

Difference within reported uncertainty; not counted as underperformance.

paper 0.9119 ±0.0004 Δ at pub 0.0001 Δ today 0.0001

TU Dortmund

#ModelAccuracy ↑Paper
1
MSH-GNN
0.9910 ±0.0030
2
ID-GNN+BEC
0.9830 ±0.0360
3
SupCosine
0.9830 ±0.0250
39
GIN
0.9330 ±0.0290
159
WL
actual #159 vs paper #596 437 places worse
0.9040 ±0.0570
best reported
actual #159 vs paper #596 437 places worse
paper 0.8072 ±0.0300
paper 0.8072 ±0.0300 Δ at pub 0.0968
199
MVGRL
actual #199 vs paper #707 508 places worse
0.8993 ±0.0616
best reported
actual #199 vs paper #707 508 places worse
paper 0.7540 ±0.0780
paper 0.7540 ±0.0780 Δ at pub 0.1453
227
MGCL
0.8955 ±0.0108
This paper
257
InfoGraph
actual #257 vs paper #272 15 places worse
0.8930 ±0.0514
best reported
actual #257 vs paper #272 15 places worse
paper 0.8901 ±0.0113

Difference within reported uncertainty; not counted as underperformance.

paper 0.8901 ±0.0113 Δ at pub 0.0029
351
JOAO
actual #351 vs paper #370 19 places worse
0.8770 ±0.0080
best reported
actual #351 vs paper #370 19 places worse
paper 0.8735 ±0.0102

Difference within reported uncertainty; not counted as underperformance.

paper 0.8735 ±0.0102 Δ at pub 0.0035 Δ today 0.0035
354
GraphCL
actual #354 vs paper #396 42 places worse
0.8766 ±0.0103
best reported
actual #354 vs paper #396 42 places worse
paper 0.8680 ±0.0134

Difference within reported uncertainty; not counted as underperformance.

paper 0.8680 ±0.0134 Δ at pub 0.0086
365
DGK
0.8744 ±0.0272
This paper
403
MLP
0.8660 ±0.0495
407
graph2vec
actual #407 vs paper #517 110 places worse
0.8644 ±0.0125
best reported
actual #407 vs paper #517 110 places worse
paper 0.8315 ±0.0925

Difference within reported uncertainty; not counted as underperformance.

paper 0.8315 ±0.0925 Δ at pub 0.0329
444
GL
actual #444 vs paper #577 133 places worse
0.8520 ±0.0090
best reported
actual #444 vs paper #577 133 places worse
paper 0.8166 ±0.0211
paper 0.8166 ±0.0211 Δ at pub 0.0354
472
Sub2Vec
actual #472 vs paper #786 314 places worse
0.8447 ±0.0037
best reported
actual #472 vs paper #786 314 places worse
paper 0.6105 ±0.1580

Difference within reported uncertainty; not counted as underperformance.

paper 0.6105 ±0.1580 Δ at pub 0.2342
746
Node2Vec
0.7263 ±0.1020
This paper
#ModelAccuracy ↑Paper
1
WL-MLP
0.9954
2
GIN
0.9934
3
GCN
0.9727
12
WL
actual #12 vs paper #223 211 places worse
0.8590 ±0.0010
best reported
actual #12 vs paper #223 211 places worse
paper 0.8001 ±0.0050
best at pub: Propagation Kernels (2014)
paper 0.8001 ±0.0050 Δ at pub 0.0589
104
GraphCL
actual #104 vs paper #325 221 places worse
0.8250 ±0.0013
best reported
actual #104 vs paper #325 221 places worse
paper 0.7787 ±0.0041
paper 0.7787 ±0.0041 Δ at pub 0.0463
131
GraphSAGE
0.8200 ±0.0100
210
DGK
0.8031 ±0.0046
This paper
283
MGCL
0.7866 ±0.0034
This paper
301
JOAO
actual #301 vs paper #317 16 places worse
0.7840 ±0.0050
best reported
actual #301 vs paper #317 16 places worse
paper 0.7807 ±0.0047

Difference within reported uncertainty; not counted as underperformance.

paper 0.7807 ±0.0047 Δ at pub 0.0033
338
InfoGraph
actual #338 vs paper #386 48 places worse
0.7750 ±0.0074
best reported
actual #338 vs paper #386 48 places worse
paper 0.7620 ±0.0106

Difference within reported uncertainty; not counted as underperformance.

paper 0.7620 ±0.0106 Δ at pub 0.0130
406
graph2vec
actual #406 vs paper #446 40 places worse
0.7540 ±0.0120
best reported
actual #406 vs paper #446 40 places worse
paper 0.7322 ±0.0181

Difference within reported uncertainty; not counted as underperformance.

paper 0.7322 ±0.0181 Δ at pub 0.0218
557
Sub2Vec
actual #557 vs paper #575 18 places worse
0.5976 ±0.0035
best reported
actual #557 vs paper #575 18 places worse
paper 0.5284 ±0.0147
paper 0.5284 ±0.0147 Δ at pub 0.0692
571
Node2Vec
actual #571 vs paper #571
0.5490 ±0.0160
best reported
actual #571 vs paper #571
paper 0.5489 ±0.0161

Difference within reported uncertainty; not counted as underperformance.

paper 0.5489 ±0.0161 Δ at pub 0.0001 Δ today 0.0001
#ModelAccuracy ↑Paper
1
MSH-GNN
0.9410 ±0.0330
2
GRDL
0.8260 ±0.0120
3
GMN
0.8225
32
GIN
0.7880 ±0.0410
149
MLP
0.7600 ±0.0320
178
DGK
actual #178 vs paper #349 171 places worse
0.7568 ±0.0054
best reported
actual #178 vs paper #349 171 places worse
paper 0.7330 ±0.0082
best today: Hierarchical Graph Capsule Network (AAAI 2020)
paper 0.7330 ±0.0082 Δ at pub 0.0238 Δ today 0.0238
190
WL
actual #190 vs paper #369 179 places worse
0.7560 ±0.0040
best reported
actual #190 vs paper #369 179 places worse
paper 0.7292 ±0.0056
paper 0.7292 ±0.0056 Δ at pub 0.0268
265
GraphCL
actual #265 vs paper #296 31 places worse
0.7489 ±0.0065
best reported
actual #265 vs paper #296 31 places worse
paper 0.7439 ±0.0045

Difference within reported uncertainty; not counted as underperformance.

paper 0.7439 ±0.0045 Δ at pub 0.0050
269
MGCL
0.7485 ±0.0071
This paper
283
JOAO
actual #283 vs paper #288 5 places worse
0.7460 ±0.0040
best reported
actual #283 vs paper #288 5 places worse
paper 0.7455 ±0.0041

Difference within reported uncertainty; not counted as underperformance.

paper 0.7455 ±0.0041 Δ at pub 0.0005 Δ today 0.0005
294
InfoGraph
0.7444 ±0.0031
This paper
350
graph2vec
0.7330 ±0.0205
This paper
474
Sub2Vec
actual #474 vs paper #517 43 places worse
0.6731 ±0.0050
best reported
actual #474 vs paper #517 43 places worse
paper 0.5303 ±0.0555
paper 0.5303 ±0.0555 Δ at pub 0.1428
514
Node2Vec
actual #514 vs paper #514
0.5750 ±0.0360
best reported
actual #514 vs paper #514
paper 0.5749 ±0.0357

Difference within reported uncertainty; not counted as underperformance.

paper 0.5749 ±0.0357 Δ at pub 0.0001
#ModelAccuracy ↑Paper
1
WL-MLP
0.9792
2
MSH-GNN
0.9640 ±0.0070
3
CoCN
0.8722 ±0.0013
23
GCN
0.8260 ±0.0220
80
GraphCL
actual #80 vs paper #290 210 places worse
0.8108 ±0.0017
best reported
actual #80 vs paper #290 210 places worse
paper 0.7136 ±0.0115
paper 0.7136 ±0.0115 Δ at pub 0.0972
146
InfoGraph
actual #146 vs paper #295 149 places worse
0.7940
best reported
actual #146 vs paper #295 149 places worse
paper 0.7065 ±0.0113
paper 0.7065 ±0.0113 Δ at pub 0.0875
151
MLP
0.7920 ±0.0240
248
JOAO
actual #248 vs paper #305 57 places worse
0.7535 ±0.0093
best reported
actual #248 vs paper #305 57 places worse
paper 0.6950 ±0.0036
paper 0.6950 ±0.0036 Δ at pub 0.0585
289
MGCL
0.7144 ±0.0071
This paper
#ModelAccuracy ↑Paper
1
OEPG
0.9630 ±0.0090
2
MSH-GNN
0.9580 ±0.0090
3
GNN-AK+
0.9480 ±0.0080
13
GIN
0.9240 ±0.0250
23
GraphCL
actual #23 vs paper #43 20 places worse
0.9169 ±0.0070
best reported
actual #23 vs paper #43 20 places worse
paper 0.8953 ±0.0084
paper 0.8953 ±0.0084 Δ at pub 0.0216
24
JOAO
actual #24 vs paper #63 39 places worse
0.9166 ±0.0059
best reported
actual #24 vs paper #63 39 places worse
paper 0.8529 ±0.0135
paper 0.8529 ±0.0135 Δ at pub 0.0637
35
MGCL
0.9025 ±0.0039
This paper
46
InfoGraph
actual #46 vs paper #68 22 places worse
0.8939 ±0.0181
best reported
actual #46 vs paper #68 22 places worse
paper 0.8250 ±0.0142
paper 0.8250 ±0.0142 Δ at pub 0.0689
65
MVGRL
actual #65 vs paper #68 3 places worse
0.8450 ±0.0060
best reported
actual #65 vs paper #68 3 places worse
paper 0.8200 ±0.0110
paper 0.8200 ±0.0110 Δ at pub 0.0250 Δ today 0.0250
73
DGK
0.7804 ±0.0039
This paper
75
GL
0.7734 ±0.0018
This paper
80
graph2vec
actual #80 vs paper #80
0.7580 ±0.0100
best reported
actual #80 vs paper #80
paper 0.7578 ±0.0103

Difference within reported uncertainty; not counted as underperformance.

paper 0.7578 ±0.0103 Δ at pub 0.0002 Δ today 0.0002
83
Sub2Vec
actual #83 vs paper #83
0.7150 ±0.0040
best reported
actual #83 vs paper #83
paper 0.7148 ±0.0041

Difference within reported uncertainty; not counted as underperformance.

paper 0.7148 ±0.0041 Δ at pub 0.0002
85
WL
0.6882 ±0.0041
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best at pub — reported value improved over the best known when published