UGMAE: A Unified Framework for Graph Masked Autoencoders

Yijun Tian, Chuxu Zhang, Ziyi Kou, Zheyuan Liu, Xiangliang Zhang, N. Chawla

Main figure for UGMAE: A Unified Framework for Graph Masked Autoencoders

Abstract

Generative self-supervised learning on graphs, particularly graph masked autoencoders, has emerged as a popular learning paradigm and demonstrated its efficacy in handling non-Euclidean data. However, several remaining issues limit the capability of existing methods: 1) the disregard of uneven node significance in masking, 2) the underutilization of holistic graph information, 3) the ignorance of semantic knowledge in the representation space due to the exclusive use of reconstruction loss in the output space, and 4) the unstable reconstructions caused by the large volume of masked contents. In light of this, we propose UGMAE, a unified framework for graph masked autoencoders to address these issues from the perspectives of adaptivity, integrity, complementarity, and consistency. Specifically, we first develop an adaptive feature mask generator to account for the unique significance of nodes and sample informative masks (adaptivity). We then design a ranking-based structure reconstruction objective joint with feature reconstruction to capture holistic graph information and emphasize the topological proximity between neighbors (integrity). After that, we present a bootstrapping-based similarity module to encode the high-level semantic knowledge in the representation space, complementary to the low-level reconstruction in the output space (complementarity). Finally, we build a consistency assurance module to provide reconstruction objectives with extra stabilized consistency targets (consistency). Extensive experiments demonstrate that UGMAE outperforms both contrastive and generative state-of-the-art baselines on several tasks across multiple datasets.

Proposed method: UGMAE

Benchmark Results

Tested on 10 datasets 29 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

#ModelF1 ↑Paper
1
GAMLP(JK)
0.9982 ±0.0001
2
GraphCON-GCN
0.9960
3
GCNII
0.9953 ±0.0001
21
GCN
actual #21 vs paper #108 87 places worse
0.9850
best reported
actual #21 vs paper #108 87 places worse
paper 0.7570 ±0.0010
best at pub: Graph-Coupled Oscillator Networks (ICML 2022)
paper 0.7570 ±0.0010 Δ at pub 0.2280
57
GAT
actual #57 vs paper #57
0.9732 ±0.0032
best reported
actual #57 vs paper #57
paper 0.9730 ±0.0020

Difference within reported uncertainty; not counted as underperformance.

paper 0.9730 ±0.0020 Δ at pub 0.0002
92
GIN
0.8724 ±0.0002
111
UGMAE
0.7483 ±0.0033
This paper
113
GraphMAE2
0.7451 ±0.0036
This paper
114
GraphMAE
0.7450 ±0.0029
This paper
117
BGRL
0.7363 ±0.0016
This paper
119
CCA-SSG
0.7334 ±0.0017
This paper
126
GRACE
0.6971 ±0.0017
This paper
148
DGI
0.6380 ±0.0020
This paper
173
MLP
0.4620

TU Dortmund

#ModelAccuracy ↑Paper
1
MSH-GNN
0.9910 ±0.0030
2
ID-GNN+BEC
0.9830 ±0.0360
3
SupCosine
0.9830 ±0.0250
39
GIN
actual #39 vs paper #237 198 places worse
0.9330 ±0.0290
best reported
actual #39 vs paper #237 198 places worse
paper 0.8940 ±0.0560

Difference within reported uncertainty; not counted as underperformance.

paper 0.8940 ±0.0560 Δ at pub 0.0390
63
GCN
0.9220 ±0.0440
105
InfoGCL
0.9120 ±0.0130
This paper
159
WL
actual #159 vs paper #596 437 places worse
0.9040 ±0.0570
best reported
actual #159 vs paper #596 437 places worse
paper 0.8072 ±0.0300
paper 0.8072 ±0.0300 Δ at pub 0.0968
199
MVGRL
actual #199 vs paper #213 14 places worse
0.8993 ±0.0616
best reported
actual #199 vs paper #213 14 places worse
paper 0.8970 ±0.0110

Difference within reported uncertainty; not counted as underperformance.

paper 0.8970 ±0.0110 Δ at pub 0.0023
226
GraphMAE2
actual #226 vs paper #402 176 places worse
0.8960 ±0.0030
best reported
actual #226 vs paper #402 176 places worse
paper 0.8663 ±0.0133
paper 0.8663 ±0.0133 Δ today 0.0297
239
DiffPool
actual #239 vs paper #447 208 places worse
0.8940 ±0.0460
best reported
actual #239 vs paper #447 208 places worse
paper 0.8500 ±0.1030
best at pub: Total Variation Graph Neural Networks (ICML 2022)

Difference within reported uncertainty; not counted as underperformance.

paper 0.8500 ±0.1030 Δ at pub 0.0140 Δ today 0.0440
257
InfoGraph
actual #257 vs paper #272 15 places worse
0.8930 ±0.0514
best reported
actual #257 vs paper #272 15 places worse
paper 0.8901 ±0.0113

Difference within reported uncertainty; not counted as underperformance.

paper 0.8901 ±0.0113 Δ at pub 0.0029
258
GraphMAE
actual #258 vs paper #326 68 places worse
0.8930 ±0.0110
best reported
actual #258 vs paper #326 68 places worse
paper 0.8819 ±0.0126

Difference within reported uncertainty; not counted as underperformance.

paper 0.8819 ±0.0126 Δ today 0.0111
324
UGMAE
0.8826 ±0.0119
This paper
351
JOAO
actual #351 vs paper #370 19 places worse
0.8770 ±0.0080
best reported
actual #351 vs paper #370 19 places worse
paper 0.8735 ±0.0102

Difference within reported uncertainty; not counted as underperformance.

paper 0.8735 ±0.0102 Δ at pub 0.0035 Δ today 0.0035
354
GraphCL
actual #354 vs paper #396 42 places worse
0.8766 ±0.0103
best reported
actual #354 vs paper #396 42 places worse
paper 0.8680 ±0.0134

Difference within reported uncertainty; not counted as underperformance.

paper 0.8680 ±0.0134 Δ at pub 0.0086
365
DGK
0.8744 ±0.0272
This paper
403
MLP
0.8660 ±0.0495
407
graph2vec
actual #407 vs paper #517 110 places worse
0.8644 ±0.0125
best reported
actual #407 vs paper #517 110 places worse
paper 0.8315 ±0.0925

Difference within reported uncertainty; not counted as underperformance.

paper 0.8315 ±0.0925 Δ at pub 0.0329
#ModelAccuracy ↑Paper
1
WL-MLP
0.9954
2
GIN
actual #2 vs paper #91 89 places worse
0.9934
best reported
actual #2 vs paper #91 89 places worse
paper 0.8270 ±0.0170
paper 0.8270 ±0.0170 Δ at pub 0.1664
3
GCN
0.9727
12
WL
actual #12 vs paper #209 197 places worse
0.8590 ±0.0010
best reported
actual #12 vs paper #209 197 places worse
paper 0.8031 ±0.0046
best at pub: Propagation Kernels (2014)
paper 0.8031 ±0.0046 Δ at pub 0.0559
104
GraphCL
actual #104 vs paper #325 221 places worse
0.8250 ±0.0013
best reported
actual #104 vs paper #325 221 places worse
paper 0.7787 ±0.0041
paper 0.7787 ±0.0041 Δ at pub 0.0463
116
GraphMAE2
actual #116 vs paper #293 177 places worse
0.8230 ±0.0090
best reported
actual #116 vs paper #293 177 places worse
paper 0.7856 ±0.0026
paper 0.7856 ±0.0026 Δ today 0.0374
131
GraphSAGE
0.8200 ±0.0100
199
UGMAE
0.8046 ±0.0017
This paper
200
GraphMAE
actual #200 vs paper #202 2 places worse
0.8042 ±0.0035
best reported
actual #200 vs paper #202 2 places worse
paper 0.8040 ±0.0030

Difference within reported uncertainty; not counted as underperformance.

paper 0.8040 ±0.0030 Δ today 0.0002
210
DGK
0.8031 ±0.0046
This paper
215
InfoGCL
0.8020 ±0.0060
This paper
301
JOAO
actual #301 vs paper #317 16 places worse
0.7840 ±0.0050
best reported
actual #301 vs paper #317 16 places worse
paper 0.7807 ±0.0047

Difference within reported uncertainty; not counted as underperformance.

paper 0.7807 ±0.0047 Δ at pub 0.0033
338
InfoGraph
actual #338 vs paper #386 48 places worse
0.7750 ±0.0074
best reported
actual #338 vs paper #386 48 places worse
paper 0.7620 ±0.0106

Difference within reported uncertainty; not counted as underperformance.

paper 0.7620 ±0.0106 Δ at pub 0.0130
406
graph2vec
actual #406 vs paper #446 40 places worse
0.7540 ±0.0120
best reported
actual #406 vs paper #446 40 places worse
paper 0.7322 ±0.0181

Difference within reported uncertainty; not counted as underperformance.

paper 0.7322 ±0.0181 Δ at pub 0.0218
#ModelAccuracy ↑Paper
1
WL-MLP
0.9792
2
MSH-GNN
0.9640 ±0.0070
3
CoCN
0.8722 ±0.0013
10
GraphMAE2
actual #10 vs paper #190 180 places worse
0.8410 ±0.0030
best reported
actual #10 vs paper #190 180 places worse
paper 0.7759 ±0.0022
paper 0.7759 ±0.0022 Δ today 0.0651
22
GraphMAE
actual #22 vs paper #109 87 places worse
0.8270 ±0.0100
best reported
actual #22 vs paper #109 87 places worse
paper 0.8032 ±0.0046
paper 0.8032 ±0.0046 Δ today 0.0238
23
GCN
0.8260 ±0.0220
30
GIN
actual #30 vs paper #111 81 places worse
0.8220 ±0.0210
best reported
actual #30 vs paper #111 81 places worse
paper 0.8020 ±0.0190

Difference within reported uncertainty; not counted as underperformance.

paper 0.8020 ±0.0190 Δ today 0.0200
33
DiffPool
actual #33 vs paper #161 128 places worse
0.8213 ±0.0043
best reported
actual #33 vs paper #161 128 places worse
paper 0.7890 ±0.0230

Difference within reported uncertainty; not counted as underperformance.

paper 0.7890 ±0.0230 Δ at pub 0.0037 Δ today 0.0323
45
UGMAE
0.8166 ±0.0012
This paper
75
GCC
actual #75 vs paper #161 86 places worse
0.8110
best reported
actual #75 vs paper #161 86 places worse
paper 0.7890
paper 0.7890 Δ at pub 0.0220
80
GraphCL
actual #80 vs paper #290 210 places worse
0.8108 ±0.0017
best reported
actual #80 vs paper #290 210 places worse
paper 0.7136 ±0.0115
paper 0.7136 ±0.0115 Δ at pub 0.0972
126
InfoGCL
actual #126 vs paper #126
0.8001 ±0.0132
best reported
actual #126 vs paper #126
paper 0.8000 ±0.0130

Difference within reported uncertainty; not counted as underperformance.

paper 0.8000 ±0.0130 Δ today 0.0001
146
InfoGraph
actual #146 vs paper #295 149 places worse
0.7940
best reported
actual #146 vs paper #295 149 places worse
paper 0.7065 ±0.0113
paper 0.7065 ±0.0113 Δ at pub 0.0875
151
MLP
0.7920 ±0.0240
248
JOAO
actual #248 vs paper #305 57 places worse
0.7535 ±0.0093
best reported
actual #248 vs paper #305 57 places worse
paper 0.6950 ±0.0036
paper 0.6950 ±0.0036 Δ at pub 0.0585

MoleculeNet

#ModelROC-AUC ↑Paper
1
HIMP
0.8740 ±0.0050
2
GINE w/ VN
0.8720 ±0.0030
3
NaiveGINe+ K=3 w/ VN
0.8700 ±0.0040
14
GCN
0.8400 ±0.0040
31
JOAO
actual #31 vs paper #231 200 places worse
0.8230 ±0.0220
best reported
actual #31 vs paper #231 200 places worse
paper 0.7500 ±0.0030
paper 0.7500 ±0.0030 Δ at pub 0.0730
37
Infomax
actual #37 vs paper #211 174 places worse
0.8160 ±0.0210
best reported
actual #37 vs paper #211 174 places worse
paper 0.7530 ±0.0050
paper 0.7530 ±0.0050 Δ at pub 0.0630
46
ContextPred
actual #46 vs paper #193 147 places worse
0.8060 ±0.0120
best reported
actual #46 vs paper #193 147 places worse
paper 0.7570 ±0.0070
paper 0.7570 ±0.0070 Δ at pub 0.0490
47
GraphCL
actual #47 vs paper #265 218 places worse
0.8050 ±0.0170
best reported
actual #47 vs paper #265 218 places worse
paper 0.7390 ±0.0070
paper 0.7390 ±0.0070 Δ at pub 0.0660
71
GraphLoG
actual #71 vs paper #193 122 places worse
0.7960 ±0.0250
best reported
actual #71 vs paper #193 122 places worse
paper 0.7570 ±0.0050
paper 0.7570 ±0.0050 Δ at pub 0.0390
163
AttrMasking
0.7670 ±0.0040
This paper
176
UGMAE
0.7630 ±0.0030
This paper
190
GraphMAE2
0.7580 ±0.0090
This paper
202
GraphMAE
0.7550 ±0.0060
This paper
255
No-pretrain
0.7430 ±0.0050
This paper
#ModelROC-AUC ↑Paper
1
ProtoMol
0.9140 ±0.0030
2
MMSG
0.9080 ±0.0050
3
MemGNN
0.9070
43
GIN
0.8646 ±0.0081
67
UGMAE
0.8530 ±0.0100
This paper
98
GraphLoG
0.8350 ±0.0120
This paper
100
JOAO
actual #100 vs paper #226 126 places worse
0.8340 ±0.0330
best reported
actual #100 vs paper #226 126 places worse
paper 0.7730 ±0.0050
paper 0.7730 ±0.0050 Δ at pub 0.0610
105
GraphMAE
0.8310 ±0.0090
This paper
151
GraphMAE2
0.8120 ±0.0130
This paper
186
ContextPred
0.7960 ±0.0120
This paper
191
AttrMasking
0.7930 ±0.0160
This paper
213
GraphCL
actual #213 vs paper #248 35 places worse
0.7820 ±0.0120
best reported
actual #213 vs paper #248 35 places worse
paper 0.7540 ±0.0140
paper 0.7540 ±0.0140 Δ at pub 0.0280
225
Infomax
actual #225 vs paper #243 18 places worse
0.7590 ±0.0160
actual #225 vs paper #243 18 places worse
paper 0.7590 ±0.0160

Improvement within reported uncertainty; not highlighted.

paper 0.7590 ±0.0160 Δ at pub 0.0030
231
No-pretrain
actual #231 vs paper #283 52 places worse
0.7700 ±0.0210
best reported
actual #231 vs paper #283 52 places worse
paper 0.7000 ±0.0250
paper 0.7000 ±0.0250 Δ today 0.0700
#ModelROC-AUC ↑Paper
1
MAGPrompt+
0.9432 ±0.0238
2
CamS-LLaMA
0.9420
3
SE(3)-I MPPH
0.9400 ±0.0210
34
GIN
0.8875 ±0.0049
36
GraphCL
actual #36 vs paper #158 122 places worse
0.8870 ±0.0190
best reported
actual #36 vs paper #158 122 places worse
paper 0.6970 ±0.0070
paper 0.6970 ±0.0070 Δ at pub 0.1900
39
JOAO
actual #39 vs paper #140 101 places worse
0.8790 ±0.0200
best reported
actual #39 vs paper #140 101 places worse
paper 0.7020 ±0.0100
paper 0.7020 ±0.0100 Δ at pub 0.1770
40
ContextPred
actual #40 vs paper #279 239 places worse
0.8770 ±0.0260
best reported
actual #40 vs paper #279 239 places worse
paper 0.6430 ±0.0280
paper 0.6430 ±0.0280 Δ at pub 0.2340
54
GraphLoG
actual #54 vs paper #90 36 places worse
0.8460 ±0.0080
best reported
actual #54 vs paper #90 36 places worse
paper 0.7250 ±0.0080
paper 0.7250 ±0.0080 Δ at pub 0.1210
55
Infomax
actual #55 vs paper #183 128 places worse
0.8400 ±0.0260
best reported
actual #55 vs paper #183 128 places worse
paper 0.6880 ±0.0080
paper 0.6880 ±0.0080 Δ at pub 0.1520
87
UGMAE
0.7280 ±0.0040
This paper
101
GraphMAE
0.7200 ±0.0060
This paper
116
GraphMAE2
0.7150 ±0.0150
This paper
230
AttrMasking
actual #230 vs paper #279 49 places worse
0.6712 ±0.0045
best reported
actual #230 vs paper #279 49 places worse
paper 0.6430 ±0.0280

Difference within reported uncertainty; not counted as underperformance.

paper 0.6430 ±0.0280 Δ today 0.0282
265
No-pretrain
0.6550 ±0.0180
This paper
#ModelROC-AUC ↑Paper
1
SE(3)-I MPPH
0.9930 ±0.0040
2
KA-GCN
0.9920 ±0.0050
3
KA-GAT
0.9910 ±0.0050
19
GCN
0.9130 ±0.0170
69
UGMAE
0.8240 ±0.0130
This paper
71
GraphMAE
0.8230 ±0.0120
This paper
79
JOAO
actual #79 vs paper #79
0.8132 ±0.0249
best reported
actual #79 vs paper #79
paper 0.8130 ±0.0250

Difference within reported uncertainty; not counted as underperformance.

paper 0.8130 ±0.0250 Δ at pub 0.0002
97
GraphCL
actual #97 vs paper #120 23 places worse
0.7890 ±0.0420
best reported
actual #97 vs paper #120 23 places worse
paper 0.7600 ±0.0270

Difference within reported uncertainty; not counted as underperformance.

paper 0.7600 ±0.0270 Δ at pub 0.0290 Δ today 0.0290
101
GraphMAE2
0.7890 ±0.0280
This paper
103
GraphLoG
actual #103 vs paper #116 13 places worse
0.7872 ±0.0258
best reported
actual #103 vs paper #116 13 places worse
paper 0.7670 ±0.0330

Difference within reported uncertainty; not counted as underperformance.

paper 0.7670 ±0.0330 Δ at pub 0.0202 Δ today 0.0202
115
ContextPred
actual #115 vs paper #197 82 places worse
0.7720 ±0.0088
best reported
actual #115 vs paper #197 82 places worse
paper 0.6590 ±0.0380
151
Infomax
actual #151 vs paper #174 23 places worse
0.7300 ±0.0320
best reported
actual #151 vs paper #174 23 places worse
paper 0.6990 ±0.0300

Difference within reported uncertainty; not counted as underperformance.

paper 0.6990 ±0.0300 Δ at pub 0.0310
162
AttrMasking
0.7180 ±0.0410
This paper
205
No-pretrain
actual #205 vs paper #228 23 places worse
0.6440 ±0.0420
best reported
actual #205 vs paper #228 23 places worse
paper 0.5820 ±0.0280

Difference within reported uncertainty; not counted as underperformance.

paper 0.5820 ±0.0280 Δ today 0.0620
#ModelROC-AUC ↑Paper
1
KA-GAT
0.8470 ±0.0020
2
KA-GCN
0.8420 ±0.0010
3
GraphKAN
0.8370 ±0.0010
56
UGMAE
0.6460 ±0.0040
This paper
66
GIN
0.6403 ±0.0104
68
JOAO
actual #68 vs paper #182 114 places worse
0.6400 ±0.0100
best reported
actual #68 vs paper #182 114 places worse
paper 0.6000 ±0.0080
paper 0.6000 ±0.0080 Δ at pub 0.0400
103
ContextPred
actual #103 vs paper #144 41 places worse
0.6220 ±0.0059
best reported
actual #103 vs paper #144 41 places worse
paper 0.6090 ±0.0060
114
GraphCL
actual #114 vs paper #162 48 places worse
0.6183 ±0.0060
best reported
actual #114 vs paper #162 48 places worse
paper 0.6050 ±0.0090

Difference within reported uncertainty; not counted as underperformance.

paper 0.6050 ±0.0090 Δ at pub 0.0030 Δ today 0.0133
120
Infomax
actual #120 vs paper #206 86 places worse
0.6160 ±0.0240
best reported
actual #120 vs paper #206 86 places worse
paper 0.5840 ±0.0080

Difference within reported uncertainty; not counted as underperformance.

paper 0.5840 ±0.0080 Δ at pub 0.0320
123
GraphLoG
actual #123 vs paper #133 10 places worse
0.6150 ±0.0130
best reported
actual #123 vs paper #133 10 places worse
paper 0.6120 ±0.0110

Difference within reported uncertainty; not counted as underperformance.

paper 0.6120 ±0.0110 Δ at pub 0.0030
133
AttrMasking
actual #133 vs paper #141 8 places worse
0.6121 ±0.0065
best reported
actual #133 vs paper #141 8 places worse
paper 0.6100 ±0.0070

Difference within reported uncertainty; not counted as underperformance.

paper 0.6100 ±0.0070 Δ today 0.0021
175
GraphMAE
0.6030 ±0.0110
This paper
193
GraphMAE2
0.5970 ±0.0060
This paper
213
No-pretrain
actual #213 vs paper #223 10 places worse
0.5810 ±0.0100
best reported
actual #213 vs paper #223 10 places worse
paper 0.5720 ±0.0070

Difference within reported uncertainty; not counted as underperformance.

paper 0.5720 ±0.0070 Δ today 0.0090
#ModelROC-AUC ↑Paper
1
GeomGCL
0.7630
2
LiGhT DeepGPT
0.7570 ±0.0110
3
CoMPT
0.7550
14
GCN
0.7350 ±0.0020
28
ContextPred
actual #28 vs paper #131 103 places worse
0.7140 ±0.0180
best reported
actual #28 vs paper #131 103 places worse
paper 0.6390 ±0.0060
paper 0.6390 ±0.0060 Δ at pub 0.0750
31
JOAO
actual #31 vs paper #165 134 places worse
0.7110 ±0.0140
best reported
actual #31 vs paper #165 134 places worse
paper 0.6290 ±0.0050
paper 0.6290 ±0.0050 Δ at pub 0.0820
42
GraphCL
actual #42 vs paper #178 136 places worse
0.6960 ±0.0230
best reported
actual #42 vs paper #178 136 places worse
paper 0.6240 ±0.0060
paper 0.6240 ±0.0060 Δ at pub 0.0720
49
Infomax
actual #49 vs paper #168 119 places worse
0.6900 ±0.0120
best reported
actual #49 vs paper #168 119 places worse
paper 0.6270 ±0.0040
paper 0.6270 ±0.0040 Δ at pub 0.0630
64
GraphLoG
actual #64 vs paper #137 73 places worse
0.6770 ±0.0190
best reported
actual #64 vs paper #137 73 places worse
paper 0.6350 ±0.0070
paper 0.6350 ±0.0070 Δ at pub 0.0420
97
UGMAE
0.6560 ±0.0030
This paper
102
GraphMAE2
0.6530 ±0.0080
This paper
126
AttrMasking
0.6420 ±0.0050
This paper
128
GraphMAE
0.6410 ±0.0030
This paper
152
No-pretrain
0.6330 ±0.0150
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best today — OK at publication; today's leaderboard best is elsewhere