KDD 2024

Revisiting Graph Autoencoders as Implicit Contrastive Learners

Jintang Li, Ruofan Wu, Yuchang Zhu, Huizhe Zhang, Zulun Zhu, Liang Chen

Main figure for Revisiting Graph Autoencoders as Implicit Contrastive Learners

Abstract

Graph autoencoders (GAEs) and graph contrastive learning (GCL) are two major paradigms for self-supervised representation learning on graphs, yet they are often studied in isolation and treated as fundamentally different approaches. In this work, we revisit GAEs through the lens of contrastive learning and show that both structure-based and feature-based GAEs can be conceptualized as implicitly graph contrastive learners. This perspective reveals that many existing GAEs differ primarily in how contrastive views are constructed, rather than in their learning objectives or architectures. Building on this insight, we introduce a unified formulation that highlights contrastive view design as a central and previously less explored dimension in GAEs. In particular, we identify asymmetric contrastive views, arising from mismatches in subgraph views, as an important yet underexplored design axis in prior GAE research. We formalize this insight within a unified framework and conduct systematic experiments on representative graph learning tasks to examine its impact on performance and efficiency. Our results show that interpreting GAEs as implicit contrastive learners offers a clearer understanding of existing models and provides practical guidance for designing effective and scalable graph autoencoders.

Benchmark Results

Tested on 9 datasets 21 baseline values reported too low Non-standard metric on 3 datasets

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

#ModelAccuracy ↑Paper
1
GraphTARIF
0.9703 ±0.0019
2
GAT
0.9669 ±0.0014
3
GAT+T
0.9660 ±0.0007
7
SAGE
0.9643 ±0.0027
151
lrGAE 6
0.9350 ±0.0030
This paper
150
GiGaMAE
0.9350 ±0.0050
This paper
161
MaskGAE
actual #161 vs paper #221 60 places worse
0.9342 ±0.0026
best reported
actual #161 vs paper #221 60 places worse
paper 0.9300 ±0.0010
164
GraphMAE
actual #164 vs paper #176 12 places worse
0.9341 ±0.0010
best reported
actual #164 vs paper #176 12 places worse
paper 0.9330 ±0.0020

Difference within reported uncertainty; not counted as underperformance.

paper 0.9330 ±0.0020 Δ at pub 0.0011
167
lrGAE 7
0.9340 ±0.0050
This paper
166
S3GCL
0.9340 ±0.0020
This paper
165
AUG-MAE
0.9340 ±0.0080
This paper
172
BGRL
actual #172 vs paper #190 18 places worse
0.9337 ±0.0021
best reported
actual #172 vs paper #190 18 places worse
paper 0.9320 ±0.0030

Difference within reported uncertainty; not counted as underperformance.

paper 0.9320 ±0.0030 Δ at pub 0.0017
186
CCA-SSG
actual #186 vs paper #205 19 places worse
0.9325 ±0.0021
best reported
actual #186 vs paper #205 19 places worse
paper 0.9310 ±0.0010

Difference within reported uncertainty; not counted as underperformance.

paper 0.9310 ±0.0010 Δ at pub 0.0015
193
lrGAE 8
0.9320 ±0.0040
This paper
192
GraphMAE2
0.9320 ±0.0050
This paper
203
GRACE
actual #203 vs paper #329 126 places worse
0.9313 ±0.0017
best reported
actual #203 vs paper #329 126 places worse
paper 0.9210 ±0.0020
paper 0.9210 ±0.0020 Δ at pub 0.0070 Δ today 0.0103
208
S2GAE
0.9310 ±0.0030
This paper
213
DGI
actual #213 vs paper #366 153 places worse
0.9309 ±0.0008
best reported
actual #213 vs paper #366 153 places worse
paper 0.9160 ±0.0020
229
GGD
actual #229 vs paper #294 65 places worse
0.9290 ±0.0020
best reported
actual #229 vs paper #294 65 places worse
paper 0.9240 ±0.0030
paper 0.9240 ±0.0030 Δ at pub 0.0050
236
SEEGERA
actual #236 vs paper #237 1 place worse
0.9281 ±0.0045
best reported
actual #236 vs paper #237 1 place worse
paper 0.9280 ±0.0050

Difference within reported uncertainty; not counted as underperformance.

paper 0.9280 ±0.0050 Δ today 0.0001
342
EPAGCL
0.9200 ±0.0030
This paper
352
GAE
actual #352 vs paper #452 100 places worse
0.9185 ±0.0006
best reported
actual #352 vs paper #452 100 places worse
paper 0.8990 ±0.0050
paper 0.8990 ±0.0050 Δ at pub 0.0195
390
MLP
0.9128 ±0.0063
529
GAE_f
0.8010 ±0.0020
This paper
#ModelAccuracy ↑Paper
1
GraphTARIF
0.9461 ±0.0017
2
GCN
0.9412 ±0.0008
3
GAT
0.9398 ±0.0022
8
SAGE
0.9351 ±0.0006
57
S2GAE
actual #57 vs paper #152 95 places worse
0.9094 ±0.0008
best reported
actual #57 vs paper #152 95 places worse
paper 0.8910 ±0.0040
paper 0.8910 ±0.0040 Δ at pub 0.0184
79
BGRL
actual #79 vs paper #196 117 places worse
0.9034 ±0.0019
best reported
actual #79 vs paper #196 117 places worse
paper 0.8830 ±0.0010
paper 0.8830 ±0.0010 Δ at pub 0.0204 Δ today 0.0204
85
GiGaMAE
actual #85 vs paper #122 37 places worse
0.9020 ±0.0045
best reported
actual #85 vs paper #122 37 places worse
paper 0.8970 ±0.0070

Difference within reported uncertainty; not counted as underperformance.

paper 0.8970 ±0.0070 Δ today 0.0050
93
GGD
actual #93 vs paper #256 163 places worse
0.9010 ±0.0090
best reported
actual #93 vs paper #256 163 places worse
paper 0.8690 ±0.0030
98
AUG-MAE
actual #98 vs paper #130 32 places worse
0.9008 ±0.0012
best reported
actual #98 vs paper #130 32 places worse
paper 0.8960 ±0.0050
paper 0.8960 ±0.0050 Δ today 0.0048
100
GraphMAE
actual #100 vs paper #116 16 places worse
0.9002 ±0.0024
best reported
actual #100 vs paper #116 16 places worse
paper 0.8980 ±0.0020

Difference within reported uncertainty; not counted as underperformance.

paper 0.8980 ±0.0020 Δ at pub 0.0008 Δ today 0.0022
111
GraphMAE2
actual #111 vs paper #130 19 places worse
0.8990 ±0.0030
best reported
actual #111 vs paper #130 19 places worse
paper 0.8960 ±0.0040

Difference within reported uncertainty; not counted as underperformance.

paper 0.8960 ±0.0040 Δ today 0.0030
117
lrGAE 7
0.8980 ±0.0020
This paper
124
lrGAE 8
0.8970 ±0.0030
This paper
132
S3GCL
0.8960 ±0.0020
This paper
139
GRACE
actual #139 vs paper #283 144 places worse
0.8953 ±0.0035
best reported
actual #139 vs paper #283 144 places worse
paper 0.8620 ±0.0020
paper 0.8620 ±0.0020 Δ at pub 0.0333 Δ today 0.0333
140
MaskGAE
0.8950 ±0.0010
This paper
153
lrGAE 6
0.8910 ±0.0030
This paper
159
CCA-SSG
actual #159 vs paper #171 12 places worse
0.8896 ±0.0013
best reported
actual #159 vs paper #171 12 places worse
paper 0.8870 ±0.0020
paper 0.8870 ±0.0020 Δ at pub 0.0004 Δ today 0.0026
189
SEEGERA
actual #189 vs paper #196 7 places worse
0.8839 ±0.0026
best reported
actual #189 vs paper #196 7 places worse
paper 0.8830 ±0.0030

Difference within reported uncertainty; not counted as underperformance.

paper 0.8830 ±0.0030 Δ today 0.0009
210
DGI
actual #210 vs paper #354 144 places worse
0.8798 ±0.0081
best reported
actual #210 vs paper #354 144 places worse
paper 0.8390 ±0.0040
paper 0.8390 ±0.0040 Δ at pub 0.0408
264
GAE
actual #264 vs paper #432 168 places worse
0.8666 ±0.0007
best reported
actual #264 vs paper #432 168 places worse
paper 0.7920 ±0.0030
paper 0.7920 ±0.0030 Δ at pub 0.0746
274
EPAGCL
0.8640 ±0.0040
This paper
320
MLP
0.8463
470
GAE_f
0.7090 ±0.0030
This paper
#ModelAccuracy ↑Paper
1
GrokFormer
0.9831 ±0.0018
2
PolyFormer
0.9808 ±0.0027
3
SGFormer
0.9796 ±0.0081
9
GCN
0.9774 ±0.0035
58
MLP
0.9597 ±0.0007
71
lrGAE 8
0.9580 ±0.0010
This paper
69
BGRL
actual #69 vs paper #169 100 places worse
0.9580 ±0.0040
best reported
actual #69 vs paper #169 100 places worse
paper 0.9470 ±0.0040
paper 0.9470 ±0.0040 Δ at pub 0.0110
77
GRACE
actual #77 vs paper #158 81 places worse
0.9574 ±0.0006
best reported
actual #77 vs paper #158 81 places worse
paper 0.9500 ±0.0010
paper 0.9500 ±0.0010 Δ at pub 0.0063 Δ today 0.0074
87
lrGAE 7
0.9570 ±0.0020
This paper
101
MaskGAE
0.9560 ±0.0020
This paper
106
CCA-SSG
actual #106 vs paper #144 38 places worse
0.9559 ±0.0007
best reported
actual #106 vs paper #144 38 places worse
paper 0.9520 ±0.0050
paper 0.9520 ±0.0050 Δ at pub 0.0039
117
GraphMAE
actual #117 vs paper #156 39 places worse
0.9550 ±0.0010
best reported
actual #117 vs paper #156 39 places worse
paper 0.9510 ±0.0020
paper 0.9510 ±0.0020 Δ at pub 0.0030 Δ today 0.0040
118
GraphMAE2
actual #118 vs paper #161 43 places worse
0.9550 ±0.0020
best reported
actual #118 vs paper #161 43 places worse
paper 0.9490 ±0.0010
paper 0.9490 ±0.0010 Δ today 0.0060
119
S2GAE
0.9550 ±0.0020
This paper
127
lrGAE 6
0.9540 ±0.0010
This paper
139
S3GCL
0.9530 ±0.0030
This paper
147
EPAGCL
0.9520 ±0.0020
This paper
148
SEEGERA
0.9520 ±0.0010
This paper
146
GAE
actual #146 vs paper #146
0.9520 ±0.0010
actual #146 vs paper #146
paper 0.9520 ±0.0010
paper 0.9520 ±0.0010 Δ at pub 0.0028
159
GGD
0.9500 ±0.0040
This paper
167
DGI
actual #167 vs paper #167
0.9480 ±0.0020
actual #167 vs paper #167
paper 0.9480 ±0.0020

Improvement within reported uncertainty; not highlighted.

paper 0.9480 ±0.0020 Δ at pub 0.0023
280
GAE_f
0.5560 ±0.0070
This paper

TU Dortmund

#ModelAccuracy ↑Paper
1
MSH-GNN
0.9910 ±0.0030
2
ID-GNN+BEC
0.9830 ±0.0360
3
SupCosine
0.9830 ±0.0250
39
GIN
0.9330 ±0.0290
137
GRACE
actual #137 vs paper #272 135 places worse
0.9065 ±0.0081
best reported
actual #137 vs paper #272 135 places worse
paper 0.8900 ±0.0110
paper 0.8900 ±0.0110 Δ at pub 0.0165
147
lrGAE 7
0.9050 ±0.0080
This paper
148
lrGAE (7)
0.9050 ±0.0080
This paper
203
lrGAE 6
0.8990 ±0.0040
This paper
210
AUG-MAE
actual #210 vs paper #210
0.8980 ±0.0130
actual #210 vs paper #210
paper 0.8980 ±0.0130
paper 0.8980 ±0.0130 Δ at pub 0.0152
218
EPAGCL
0.8970 ±0.0130
This paper
226
GraphMAE2
actual #226 vs paper #227 1 place worse
0.8950 ±0.0150
actual #226 vs paper #227 1 place worse
paper 0.8950 ±0.0150
paper 0.8950 ±0.0150 Δ at pub 0.0287
231
CCA-SSG
0.8950 ±0.0070
This paper
228
GAE
actual #228 vs paper #228
0.8950 ±0.0150
actual #228 vs paper #228
paper 0.8950 ±0.0150
paper 0.8950 ±0.0150 Δ at pub 0.0180
243
BGRL
actual #243 vs paper #243
0.8940 ±0.0100
actual #243 vs paper #243
paper 0.8940 ±0.0100
paper 0.8940 ±0.0100 Δ at pub 0.0340
260
lrGAE 8
0.8930 ±0.0070
This paper
258
GraphMAE
actual #258 vs paper #258
0.8930 ±0.0110
actual #258 vs paper #258
paper 0.8930 ±0.0110
paper 0.8930 ±0.0110 Δ at pub 0.0111
266
S2GAE
actual #266 vs paper #429 163 places worse
0.8927 ±0.0153
best reported
actual #266 vs paper #429 163 places worse
paper 0.8570 ±0.0080
282
SEEGERA
0.8890 ±0.0120
This paper
304
MaskGAE
0.8860 ±0.0060
This paper
394
GAE_f
0.8700 ±0.0120
This paper
403
MLP
0.8660 ±0.0495
406
S3GCL
0.8650 ±0.0090
This paper
440
GGD
0.8540 ±0.0030
This paper
517
DGI
0.8320 ±0.0130
This paper
#ModelAccuracy ↑Paper
1
WL-MLP
0.9954
2
GIN
0.9934
3
GCN
0.9727
112
S2GAE
actual #112 vs paper #350 238 places worse
0.8237 ±0.0224
best reported
actual #112 vs paper #350 238 places worse
paper 0.7720 ±0.0080
116
GraphMAE2
actual #116 vs paper #116
0.8230 ±0.0090
actual #116 vs paper #116
paper 0.8230 ±0.0090
paper 0.8230 ±0.0090 Δ at pub 0.0374
121
MaskGAE
0.8220 ±0.0040
This paper
131
GraphSAGE
0.8200 ±0.0100
150
lrGAE 6
0.8150 ±0.0060
This paper
156
lrGAE 7
0.8140 ±0.0040
This paper
171
lrGAE 8
0.8120 ±0.0070
This paper
194
EPAGCL
0.8050 ±0.0040
This paper
200
GraphMAE
actual #200 vs paper #220 20 places worse
0.8042 ±0.0035
best reported
actual #200 vs paper #220 20 places worse
paper 0.8010 ±0.0100

Difference within reported uncertainty; not counted as underperformance.

paper 0.8010 ±0.0100 Δ at pub 0.0030 Δ today 0.0032
236
GAE
actual #236 vs paper #440 204 places worse
0.7978 ±0.0017
best reported
actual #236 vs paper #440 204 places worse
paper 0.7380 ±0.0170
paper 0.7380 ±0.0170 Δ at pub 0.0598
243
S3GCL
0.7960 ±0.0050
This paper
251
BGRL
0.7940 ±0.0060
This paper
275
CCA-SSG
0.7880 ±0.0050
This paper
276
AUG-MAE
0.7880 ±0.0120
This paper
334
SEEGERA
0.7760 ±0.0120
This paper
351
GRACE
actual #351 vs paper #351
0.7720 ±0.0150
actual #351 vs paper #351
paper 0.7720 ±0.0150

Improvement within reported uncertainty; not highlighted.

paper 0.7720 ±0.0150 Δ at pub 0.0104
374
GGD
0.7650 ±0.0030
This paper
388
DGI
0.7620 ±0.0180
This paper
422
GAE_f
0.7450 ±0.0130
This paper
#ModelAccuracy ↑Paper
1
WL-MLP
0.9792
2
MSH-GNN
0.9640 ±0.0070
3
CoCN
0.8722 ±0.0013
10
GraphMAE2
actual #10 vs paper #42 32 places worse
0.8170 ±0.0080
actual #10 vs paper #42 32 places worse
paper 0.8170 ±0.0080
paper 0.8170 ±0.0080 Δ at pub 0.0411 Δ today 0.0240
22
GraphMAE
actual #22 vs paper #22
0.8270 ±0.0100
actual #22 vs paper #22
paper 0.8270 ±0.0100
paper 0.8270 ±0.0100 Δ at pub 0.0238
23
GCN
0.8260 ±0.0220
27
S2GAE
actual #27 vs paper #29 2 places worse
0.8220 ±0.0040
actual #27 vs paper #29 2 places worse
paper 0.8220 ±0.0040
28
lrGAE 6
0.8230 ±0.0070
This paper
29
lrGAE 7
0.8230 ±0.0040
This paper
31
lrGAE 8
0.8220 ±0.0020
This paper
35
MaskGAE
0.8200 ±0.0050
This paper
48
BGRL
0.8150 ±0.0070
This paper
89
SEEGERA
0.8080 ±0.0060
This paper
91
GAE
actual #91 vs paper #118 27 places worse
0.8078 ±0.0012
best reported
actual #91 vs paper #118 27 places worse
paper 0.8010 ±0.0060
paper 0.8010 ±0.0060 Δ at pub 0.0068
95
S3GCL
0.8070 ±0.0140
This paper
103
AUG-MAE
actual #103 vs paper #129 26 places worse
0.8050 ±0.0050
best reported
actual #103 vs paper #129 26 places worse
paper 0.7990 ±0.0040
paper 0.7990 ±0.0040 Δ at pub 0.0058 Δ today 0.0060
141
GRACE
actual #141 vs paper #141
0.7960 ±0.0130
actual #141 vs paper #141
paper 0.7960 ±0.0130
paper 0.7960 ±0.0130 Δ at pub 0.0428
147
CCA-SSG
0.7940 ±0.0060
This paper
148
EPAGCL
0.7940 ±0.0130
This paper
151
MLP
0.7920 ±0.0240
212
GAE_f
0.7690 ±0.0150
This paper
224
GGD
0.7630 ±0.0120
This paper
295
DGI
0.7070 ±0.0140
This paper

Heterogeneous Graph Benchmarks

DBLP

Global leaderboard →
Non-default metric. This paper reports Accuracy; the dataset default is Micro-F1. Values below use Accuracy only.

Sorted by Accuracy. Highlighted rows are from other papers.

#ModelAccuracy ↑Paper
1
HG-SCM
0.9490 ±0.0015
2
SeHGNN
0.9489 ±0.0018
3
AdvSynGNN
0.9486 ±0.0012
4
S2GAE
0.9480 ±0.0040
This paper
5
lrGAE 7
0.9460 ±0.0030
This paper
7
lrGAE 6
0.9440 ±0.0020
This paper
8
lrGAE 8
0.9430 ±0.0020
This paper
9
SHGN
0.9420 ±0.0031
This paper
10
MaskGAE
0.9420 ±0.0020
This paper
11
GAE
actual #11 vs paper #11
0.9410 ±0.0030
actual #11 vs paper #11
paper 0.9410 ±0.0030
paper 0.9410 ±0.0030 Δ at pub 0.1290
15
HGT
0.9349 ±0.0025
This paper
19
RGCN
0.9207 ±0.0050
This paper
20
HAN
0.9205 ±0.0062
This paper
36
GCN
0.8758 ±0.0020
65
MLP
0.8118 ±0.0020

ACM

Global leaderboard →
Non-default metric. This paper reports Accuracy; the dataset default is Micro-F1. Values below use Accuracy only.

Sorted by Accuracy. Highlighted rows are from other papers.

#ModelAccuracy ↑Paper
1
SignGT
0.9545 ±0.0049
2
NodeFormer
0.9493 ±0.0035
3
FAGCN
0.9487 ±0.0059
8
lrGAE 8
0.9410 ±0.0010
This paper
11
lrGAE 7
0.9400 ±0.0030
This paper
13
S2GAE
0.9380 ±0.0070
This paper
15
MaskGAE
0.9370 ±0.0020
This paper
18
lrGAE 6
0.9350 ±0.0040
This paper
20
SHGN
0.9335 ±0.0045
This paper
24
GAE
0.9330 ±0.0050
This paper
34
GCN
0.9233 ±0.0038
35
HAN
actual #35 vs paper #44 9 places worse
0.9220 ±0.0020
best reported
actual #35 vs paper #44 9 places worse
paper 0.9079 ±0.0043
paper 0.9079 ±0.0043 Δ at pub 0.0141
36
HGT
actual #36 vs paper #41 5 places worse
0.9190 ±0.0030
best reported
actual #36 vs paper #41 5 places worse
paper 0.9115 ±0.0071

Difference within reported uncertainty; not counted as underperformance.

paper 0.9115 ±0.0071 Δ at pub 0.0075
38
RGCN
actual #38 vs paper #38
0.9175 ±0.0035
actual #38 vs paper #38
paper 0.9175 ±0.0035

Improvement within reported uncertainty; not highlighted.

paper 0.9175 ±0.0035 Δ at pub 0.0034

IMDB

Global leaderboard →
Non-default metric. This paper reports Accuracy; the dataset default is Micro-F1. Values below use Accuracy only.

Sorted by Accuracy. Highlighted rows are from other papers.

#ModelAccuracy ↑Paper
1
BERT-Base, Uncased
0.9190
2
TAGs (70b)
0.7157 ±0.0042
3
UniMP
0.6998 ±0.0058
4
GIN
0.6759 ±0.0041
5
SHGN
0.6736 ±0.0057
This paper
7
HGT
0.6720 ±0.0057
This paper
9
lrGAE 8
0.6630 ±0.0040
This paper
10
lrGAE 6
0.6600 ±0.0050
This paper
12
MaskGAE
0.6590 ±0.0070
This paper
13
S2GAE
0.6570 ±0.0050
This paper
14
GAE
0.6540 ±0.0020
This paper
15
RGCN
actual #15 vs paper #15
0.6521 ±0.0073
actual #15 vs paper #15
paper 0.6521 ±0.0073
16
lrGAE 7
0.6520 ±0.0060
This paper
19
HAN
0.6463 ±0.0058
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best at pub — reported value improved over the best known when published Improved at pub — later paper surpassed this result Best today — OK at publication; today's leaderboard best is elsewhere