ICML 2024

Community-Invariant Graph Contrastive Learning

Shiyin Tan, Dongyuan Li, Renhe Jiang, Ying Zhang, Manabu Okumura

Main figure for Community-Invariant Graph Contrastive Learning

Abstract

Graph augmentation has received great attention in recent years for graph contrastive learning (GCL) to learn well-generalized node/graph representations. However, mainstream GCL methods often favor randomly disrupting graphs for augmentation, which shows limited generalization and inevitably leads to the corruption of high-level graph information, i.e., the graph community. Moreover, current knowledge-based graph augmentation methods can only focus on either topology or node features, causing the model to lack robustness against various types of noise. To address these limitations, this research investigated the role of the graph community in graph augmentation and figured out its crucial advantage for learnable graph augmentation. Based on our observations, we propose a community-invariant GCL framework to maintain graph community structure during learnable graph augmentation. By maximizing the spectral changes, this framework unifies the constraints of both topology and feature augmentation, enhancing the model's robustness. Empirical evidence on 21 benchmark datasets demonstrates the exclusive merits of our framework. Code is released on Github (https://github.com/ShiyinTan/CI-GCL.git).

Proposed method: CI-GCL

Benchmark Results

Tested on 12 datasets 18 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

TU Dortmund

#ModelAccuracy ↑Paper
1
MSH-GNN
0.9910 ±0.0030
2
ID-GNN+BEC
0.9830 ±0.0360
3
SupCosine
0.9830 ±0.0250
39
GIN
0.9330 ±0.0290
155
GCS
actual #155 vs paper #326 171 places worse
0.9045 ±0.0081
best reported
actual #155 vs paper #326 171 places worse
paper 0.8819 ±0.0090
paper 0.8819 ±0.0090 Δ today 0.0226
178
AD-GCL
actual #178 vs paper #213 35 places worse
0.9028 ±0.0401
best reported
actual #178 vs paper #213 35 places worse
paper 0.8970 ±0.0100

Difference within reported uncertainty; not counted as underperformance.

paper 0.8970 ±0.0100 Δ at pub 0.0058
179
SEGA
0.9021 ±0.0070
This paper
209
CI+AD-GCL
0.8988 ±0.0070
This paper
223
CI-GCL
0.8967 ±0.0090
This paper
257
InfoGraph
actual #257 vs paper #272 15 places worse
0.8930 ±0.0514
best reported
actual #257 vs paper #272 15 places worse
paper 0.8901 ±0.0110

Difference within reported uncertainty; not counted as underperformance.

paper 0.8901 ±0.0110 Δ at pub 0.0029
302
AutoGCL
actual #302 vs paper #446 144 places worse
0.8864 ±0.0108
best reported
actual #302 vs paper #446 144 places worse
paper 0.8515 ±0.0110
351
JOAO
actual #351 vs paper #370 19 places worse
0.8770 ±0.0080
best reported
actual #351 vs paper #370 19 places worse
paper 0.8735 ±0.0100

Difference within reported uncertainty; not counted as underperformance.

paper 0.8735 ±0.0100 Δ at pub 0.0035 Δ today 0.0035
400
CI+AutoGCL
0.8673 ±0.0100
This paper
403
MLP
0.8660 ±0.0495
451
GCL-SPAN
0.8500 ±0.0080
This paper
#ModelAccuracy ↑Paper
1
WL-MLP
0.9954
2
GIN
0.9934
3
GCN
0.9727
104
GraphCL
actual #104 vs paper #325 221 places worse
0.8250 ±0.0013
best reported
actual #104 vs paper #325 221 places worse
paper 0.7787 ±0.0040
paper 0.7787 ±0.0040 Δ at pub 0.0463
133
AutoGCL
actual #133 vs paper #304 171 places worse
0.8200 ±0.0029
best reported
actual #133 vs paper #304 171 places worse
paper 0.7832 ±0.0050
131
GraphSAGE
0.8200 ±0.0100
195
CI-GCL
0.8050 ±0.0050
This paper
264
SEGA
0.7900 ±0.0070
This paper
296
CI+AutoGCL
0.7847 ±0.0070
This paper
301
JOAO
actual #301 vs paper #317 16 places worse
0.7840 ±0.0050
best reported
actual #301 vs paper #317 16 places worse
paper 0.7807 ±0.0040

Difference within reported uncertainty; not counted as underperformance.

paper 0.7807 ±0.0040 Δ at pub 0.0033
338
InfoGraph
actual #338 vs paper #386 48 places worse
0.7750 ±0.0074
best reported
actual #338 vs paper #386 48 places worse
paper 0.7620 ±0.0100

Difference within reported uncertainty; not counted as underperformance.

paper 0.7620 ±0.0100 Δ at pub 0.0130
344
GCS
actual #344 vs paper #353 9 places worse
0.7737 ±0.0030
best reported
actual #344 vs paper #353 9 places worse
paper 0.7718 ±0.0030

Difference within reported uncertainty; not counted as underperformance.

paper 0.7718 ±0.0030 Δ today 0.0019
356
MVGRL
actual #356 vs paper #371 15 places worse
0.7700 ±0.0080
best reported
actual #356 vs paper #371 15 places worse
paper 0.7664 ±0.0030

Difference within reported uncertainty; not counted as underperformance.

paper 0.7664 ±0.0030 Δ at pub 0.0036 Δ today 0.0036
405
GCL-SPAN
0.7543 ±0.0040
This paper
427
CI+AD-GCL
0.7435 ±0.0050
This paper
#ModelAccuracy ↑Paper
1
MSH-GNN
0.9410 ±0.0330
2
GRDL
0.8260 ±0.0120
3
GMN
0.8225
32
GIN
0.7880 ±0.0410
107
CI-GCL
0.7650 ±0.0010
This paper
146
SEGA
0.7601 ±0.0040
This paper
149
MLP
0.7600 ±0.0320
169
GCL-SPAN
0.7578 ±0.0040
This paper
245
GCS
actual #245 vs paper #309 64 places worse
0.7502 ±0.0081
best reported
actual #245 vs paper #309 64 places worse
paper 0.7404 ±0.0040

Difference within reported uncertainty; not counted as underperformance.

paper 0.7404 ±0.0040 Δ today 0.0098
251
MVGRL
actual #251 vs paper #309 58 places worse
0.7500 ±0.0290
best reported
actual #251 vs paper #309 58 places worse
paper 0.7402 ±0.0030

Difference within reported uncertainty; not counted as underperformance.

paper 0.7402 ±0.0030 Δ today 0.0098
265
GraphCL
actual #265 vs paper #296 31 places worse
0.7489 ±0.0065
best reported
actual #265 vs paper #296 31 places worse
paper 0.7439 ±0.0040

Difference within reported uncertainty; not counted as underperformance.

paper 0.7439 ±0.0040 Δ at pub 0.0050
280
CI+AD-GCL
0.7466 ±0.0060
This paper
283
JOAO
actual #283 vs paper #288 5 places worse
0.7460 ±0.0040
best reported
actual #283 vs paper #288 5 places worse
paper 0.7455 ±0.0040

Difference within reported uncertainty; not counted as underperformance.

paper 0.7455 ±0.0040 Δ at pub 0.0005 Δ today 0.0005
294
InfoGraph
0.7444 ±0.0030
This paper
444
CI+AutoGCL
0.7081 ±0.0050
This paper
#ModelAccuracy ↑Paper
1
WL-MLP
0.9792
2
MSH-GNN
0.9640 ±0.0070
3
CoCN
0.8722 ±0.0013
23
GCN
0.8260 ±0.0220
146
InfoGraph
actual #146 vs paper #295 149 places worse
0.7940
best reported
actual #146 vs paper #295 149 places worse
paper 0.7065 ±0.0110
paper 0.7065 ±0.0110 Δ at pub 0.0875
151
MLP
0.7920 ±0.0240
223
GCL-SPAN
actual #223 vs paper #289 66 places worse
0.7637 ±0.0073
best reported
actual #223 vs paper #289 66 places worse
paper 0.7140 ±0.0050
paper 0.7140 ±0.0050 Δ at pub 0.0497
239
GCS
actual #239 vs paper #262 23 places worse
0.7556 ±0.0041
best reported
actual #239 vs paper #262 23 places worse
paper 0.7400 ±0.0040
paper 0.7400 ±0.0040 Δ today 0.0156
264
CI+AD-GCL
0.7394 ±0.0030
This paper
283
CI+AutoGCL
0.7224 ±0.0090
This paper
#ModelAccuracy ↑Paper
1
OEPG
0.9630 ±0.0090
2
MSH-GNN
0.9580 ±0.0090
3
GNN-AK+
0.9480 ±0.0080
13
GIN
0.9240 ±0.0250
23
GraphCL
actual #23 vs paper #43 20 places worse
0.9169 ±0.0070
best reported
actual #23 vs paper #43 20 places worse
paper 0.8953 ±0.0080
paper 0.8953 ±0.0080 Δ at pub 0.0216
30
CI-GCL
0.9080 ±0.0050
This paper
36
SEGA
0.9021 ±0.0070
This paper
47
AutoGCL
actual #47 vs paper #56 9 places worse
0.8931 ±0.0148
best reported
actual #47 vs paper #56 9 places worse
paper 0.8660 ±0.0150
paper 0.8660 ±0.0150 Δ at pub 0.0271
54
CI+AD-GCL
0.8780 ±0.0120
This paper
56
CI+AutoGCL
0.8750 ±0.0140
This paper
57
GCS
0.8650 ±0.0030
This paper
58
GCL-SPAN
0.8650 ±0.0010
This paper

MoleculeNet

#ModelROC-AUC ↑Paper
1
HIMP
0.8740 ±0.0050
2
GINE w/ VN
0.8720 ±0.0030
3
NaiveGINe+ K=3 w/ VN
0.8700 ±0.0040
14
GCN
0.8400 ±0.0040
31
JOAO
actual #31 vs paper #293 262 places worse
0.8230 ±0.0220
best reported
actual #31 vs paper #293 262 places worse
paper 0.7183 ±0.0092
paper 0.7183 ±0.0092 Δ at pub 0.1047
47
GraphCL
actual #47 vs paper #287 240 places worse
0.8050 ±0.0170
best reported
actual #47 vs paper #287 240 places worse
paper 0.7240 ±0.0101
paper 0.7240 ±0.0101 Δ at pub 0.0810
81
InfoGraph
actual #81 vs paper #305 224 places worse
0.7930
best reported
actual #81 vs paper #305 224 places worse
paper 0.6974 ±0.0057
paper 0.6974 ±0.0057 Δ at pub 0.0956
165
SEGA
0.7670 ±0.0040
This paper
204
AutoGCL
0.7550 ±0.0030
This paper
240
AD-GCL
actual #240 vs paper #295 55 places worse
0.7470 ±0.0110
best reported
actual #240 vs paper #295 55 places worse
paper 0.7142 ±0.0073
paper 0.7142 ±0.0073 Δ at pub 0.0328
250
GCS
0.7440 ±0.0040
This paper
264
No Pre-train
0.7400 ±0.0080
This paper
272
CI-GCL
0.7359 ±0.0066
This paper
286
GCL-SPAN
0.7283 ±0.0062
This paper
301
MVGRL
0.7048 ±0.0083
This paper
#ModelROC-AUC ↑Paper
1
ProtoMol
0.9140 ±0.0030
2
MMSG
0.9080 ±0.0050
3
MemGNN
0.9070
43
GIN
0.8646 ±0.0081
87
CI-GCL
0.8440 ±0.0090
This paper
100
JOAO
actual #100 vs paper #226 126 places worse
0.8340 ±0.0330
best reported
actual #100 vs paper #226 126 places worse
paper 0.7730 ±0.0050
paper 0.7730 ±0.0050 Δ at pub 0.0610
124
GCS
0.8230 ±0.0030
This paper
177
GCL-SPAN
0.7990 ±0.0070
This paper
202
AD-GCL
actual #202 vs paper #202
0.7900 ±0.0080
actual #202 vs paper #202
paper 0.7900 ±0.0080
paper 0.7900 ±0.0080 Δ at pub 0.0270
228
MVGRL
0.7720 ±0.0100
This paper
229
SEGA
0.7710 ±0.0050
This paper
235
AutoGCL
0.7670 ±0.0110
This paper
281
No Pre-train
0.7010 ±0.0540
This paper
#ModelROC-AUC ↑Paper
1
MAGPrompt+
0.9432 ±0.0238
2
CamS-LLaMA
0.9420
3
SE(3)-I MPPH
0.9400 ±0.0210
34
GIN
0.8875 ±0.0049
70
CI-GCL
0.7440 ±0.0190
This paper
91
GCS
0.7250 ±0.0050
This paper
102
AutoGCL
0.7200 ±0.0060
This paper
107
SEGA
0.7190 ±0.0110
This paper
144
AD-GCL
actual #144 vs paper #223 79 places worse
0.7010 ±0.0120
best reported
actual #144 vs paper #223 79 places worse
paper 0.6740 ±0.0100
paper 0.6740 ±0.0100 Δ at pub 0.0270
152
GCL-SPAN
0.7000 ±0.0070
This paper
180
MVGRL
0.6900 ±0.0050
This paper
#ModelROC-AUC ↑Paper
1
SE(3)-I MPPH
0.9930 ±0.0040
2
KA-GCN
0.9920 ±0.0050
3
KA-GAT
0.9910 ±0.0050
19
GCN
0.9130 ±0.0170
58
SEGA
0.8500 ±0.0090
This paper
79
JOAO
actual #79 vs paper #104 25 places worse
0.8132 ±0.0249
best reported
actual #79 vs paper #104 25 places worse
paper 0.7821 ±0.0412

Difference within reported uncertainty; not counted as underperformance.

paper 0.7821 ±0.0412 Δ at pub 0.0311
82
AD-GCL
actual #82 vs paper #82
0.8077 ±0.0392
actual #82 vs paper #82
paper 0.8077 ±0.0392

Improvement within reported uncertainty; not highlighted.

paper 0.8077 ±0.0392 Δ at pub 0.0267
86
GCL-SPAN
0.8028 ±0.0242
This paper
90
AutoGCL
0.7990 ±0.0330
This paper
100
CI-GCL
0.7890 ±0.0259
This paper
97
GraphCL
actual #97 vs paper #130 33 places worse
0.7890 ±0.0420
best reported
actual #97 vs paper #130 33 places worse
paper 0.7492 ±0.0442

Difference within reported uncertainty; not counted as underperformance.

paper 0.7492 ±0.0442 Δ at pub 0.0398 Δ today 0.0398
106
InfoGraph
actual #106 vs paper #204 98 places worse
0.7810
best reported
actual #106 vs paper #204 98 places worse
paper 0.6450 ±0.0532
paper 0.6450 ±0.0532 Δ at pub 0.1360
141
MVGRL
0.7384 ±0.0425
This paper
195
GCS
0.6670 ±0.0190
This paper
232
No Pre-train
0.5800 ±0.0440
This paper
#ModelROC-AUC ↑Paper
1
KA-GAT
0.8470 ±0.0020
2
KA-GCN
0.8420 ±0.0010
3
GraphKAN
0.8370 ±0.0010
51
GCL-SPAN
0.6487 ±0.0088
This paper
55
CI-GCL
0.6470 ±0.0030
This paper
66
GIN
0.6403 ±0.0104
68
JOAO
actual #68 vs paper #90 22 places worse
0.6400 ±0.0100
best reported
actual #68 vs paper #90 22 places worse
paper 0.6273 ±0.0092

Difference within reported uncertainty; not counted as underperformance.

paper 0.6273 ±0.0092 Δ at pub 0.0127
74
SEGA
0.6370 ±0.0030
This paper
98
AutoGCL
0.6250 ±0.0060
This paper
103
ContextPred
actual #103 vs paper #144 41 places worse
0.6220 ±0.0059
best reported
actual #103 vs paper #144 41 places worse
paper 0.6090 ±0.0060
110
MVGRL
0.6194 ±0.0094
This paper
111
GCS
0.6190 ±0.0040
This paper
114
GraphCL
actual #114 vs paper #116 2 places worse
0.6176 ±0.0111
actual #114 vs paper #116 2 places worse
paper 0.6176 ±0.0111

Improvement within reported uncertainty; not highlighted.

paper 0.6176 ±0.0111 Δ at pub 0.0096
120
Infomax
actual #120 vs paper #206 86 places worse
0.6160 ±0.0240
best reported
actual #120 vs paper #206 86 places worse
paper 0.5840 ±0.0080

Difference within reported uncertainty; not counted as underperformance.

paper 0.5840 ±0.0080 Δ at pub 0.0320
133
AttrMasking
actual #133 vs paper #141 8 places worse
0.6121 ±0.0065
best reported
actual #133 vs paper #141 8 places worse
paper 0.6100 ±0.0070

Difference within reported uncertainty; not counted as underperformance.

paper 0.6100 ±0.0070 Δ today 0.0021
146
AD-GCL
actual #146 vs paper #146
0.6080 ±0.0090
actual #146 vs paper #146
paper 0.6080 ±0.0090
paper 0.6080 ±0.0090 Δ at pub 0.0140
162
InfoGraph
actual #162 vs paper #162
0.6054 ±0.0090
actual #162 vs paper #162
paper 0.6054 ±0.0090

Improvement within reported uncertainty; not highlighted.

paper 0.6054 ±0.0090 Δ at pub 0.0064
170
EdgePred
0.6040 ±0.0070
This paper
222
No Pre-train
0.5730 ±0.0160
This paper
#ModelROC-AUC ↑Paper
1
GeomGCL
0.7630
2
LiGhT DeepGPT
0.7570 ±0.0110
3
CoMPT
0.7550
14
GCN
0.7350 ±0.0020
31
JOAO
actual #31 vs paper #165 134 places worse
0.7110 ±0.0140
best reported
actual #31 vs paper #165 134 places worse
paper 0.6290 ±0.0050
paper 0.6290 ±0.0050 Δ at pub 0.0820
100
CI-GCL
0.6540 ±0.0150
This paper
104
SEGA
0.6520 ±0.0090
This paper
118
GCS
0.6440 ±0.0020
This paper
127
GCL-SPAN
0.6420 ±0.0040
This paper
138
AD-GCL
actual #138 vs paper #138
0.6350 ±0.0070
actual #138 vs paper #138
paper 0.6350 ±0.0070

Improvement within reported uncertainty; not highlighted.

paper 0.6350 ±0.0070 Δ at pub 0.0040
142
No Pre-train
0.6340 ±0.0060
This paper
143
AutoGCL
0.6340 ±0.0040
This paper
176
MVGRL
0.6260 ±0.0050
This paper
#ModelRMSE ↓Paper
1
SE(3)-I MPPH
0.3540 ±0.0530
2
ECFP-4 + RF
0.5600 ±0.0660
3
GPSE + GPS
0.6820
56
InfoGraph
actual #56 vs paper #128 72 places worse
2.1040
best reported
actual #56 vs paper #128 72 places worse
paper 10.0000 ±4.8200
paper 10.0000 ±4.8200 Δ at pub 7.8960
69
GIN
2.3460 ±0.1220
91
CI-GCL
2.8730 ±0.3200
This paper
101
GraphCL
actual #101 vs paper #125 24 places worse
3.1660 ±0.0270
best reported
actual #101 vs paper #125 24 places worse
paper 7.6790 ±2.7500
104
JOAO
actual #104 vs paper #123 19 places worse
3.2820 ±0.0020
best reported
actual #104 vs paper #123 19 places worse
paper 5.1310 ±0.7200
121
GCL-SPAN
4.5310 ±0.4600
This paper
124
AD-GCL
5.1500 ±0.6200
This paper
128
MVGRL
9.0240 ±1.9800
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best at pub — reported value improved over the best known when published Best today — OK at publication; today's leaderboard best is elsewhere