Graph Contrastive Learning with Personalized Augmentation

X. Zhang, Qiaoyu Tan, Xiao Huang, Bo Li

Main figure for Graph Contrastive Learning with Personalized Augmentation

Abstract

Graph contrastive learning (GCL) has emerged as an effective tool for learning unsupervised representations of graphs. The key idea is to maximize the agreement between two augmented views of each graph via data augmentation. Existing GCL models mainly focus on applying identical augmentation strategies for all graphs within a given scenario. However, real-world graphs are often not monomorphic but abstractions of diverse natures. Even within the same scenario (e.g., macromolecules and online communities), different graphs might need diverse augmentations to perform effective GCL. Thus, blindly augmenting all graphs without considering their individual characteristics may undermine the performance of GCL arts. To deal with this, we propose the first principled framework, termed as Graph contrastive learning with Personalized Augmentation (GPA), to advance conventional GCL by allowing each graph to choose its own suitable augmentation operations. In essence, GPA infers tailored augmentation strategies for each graph based on its topology and node attributes via a learnable augmentation selector, which is a plug-and-play module and can be effectively trained with downstream GCL models end-to-end. Extensive experiments across 11 benchmark graphs from different types and domains demonstrate the superiority of GPA against state-of-the-art competitors. Moreover, by visualizing the learned augmentation distributions across different types of datasets, we show that GPA can effectively identify the most suitable augmentations for each graph based on its characteristics.

Proposed method: GPA

Benchmark Results

Tested on 5 datasets 9 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

TU Dortmund

#ModelAccuracy ↑Paper
1
MSH-GNN
0.9910 ±0.0030
2
ID-GNN+BEC
0.9830 ±0.0360
3
SupCosine
0.9830 ±0.0250
39
GIN
0.9330 ±0.0290
178
AD-GCL
actual #178 vs paper #303 125 places worse
0.9028 ±0.0401
best reported
actual #178 vs paper #303 125 places worse
paper 0.8862 ±0.0127

Difference within reported uncertainty; not counted as underperformance.

paper 0.8862 ±0.0127 Δ at pub 0.0108 Δ today 0.0166
221
GPA
0.8968 ±0.0080
This paper
222
JOAOv2
actual #222 vs paper #353 131 places worse
0.8967 ±0.0524
best reported
actual #222 vs paper #353 131 places worse
paper 0.8767 ±0.0079
best at pub: Graph Contrastive Learning Automated (ICML 2021)

Difference within reported uncertainty; not counted as underperformance.

paper 0.8767 ±0.0079 Δ today 0.0200
257
InfoGraph
actual #257 vs paper #272 15 places worse
0.8930 ±0.0514
best reported
actual #257 vs paper #272 15 places worse
paper 0.8901 ±0.0113
best at pub: Graph Contrastive Learning Automated (ICML 2021)

Difference within reported uncertainty; not counted as underperformance.

paper 0.8901 ±0.0113 Δ today 0.0029
351
JOAO
actual #351 vs paper #370 19 places worse
0.8770 ±0.0080
best reported
actual #351 vs paper #370 19 places worse
paper 0.8735 ±0.0102

Difference within reported uncertainty; not counted as underperformance.

paper 0.8735 ±0.0102 Δ at pub 0.0035 Δ today 0.0035
354
GraphCL
actual #354 vs paper #396 42 places worse
0.8766 ±0.0103
best reported
actual #354 vs paper #396 42 places worse
paper 0.8680 ±0.0134

Difference within reported uncertainty; not counted as underperformance.

paper 0.8680 ±0.0134 Δ at pub 0.0086
403
MLP
0.8660 ±0.0495
#ModelAccuracy ↑Paper
1
WL-MLP
0.9954
2
GIN
0.9934
3
GCN
0.9727
104
GraphCL
actual #104 vs paper #325 221 places worse
0.8250 ±0.0013
best reported
actual #104 vs paper #325 221 places worse
paper 0.7787 ±0.0041
paper 0.7787 ±0.0041 Δ at pub 0.0463
131
GraphSAGE
0.8200 ±0.0100
146
JOAOv2
actual #146 vs paper #303 157 places worse
0.8170 ±0.0026
best reported
actual #146 vs paper #303 157 places worse
paper 0.7836 ±0.0053
paper 0.7836 ±0.0053 Δ at pub 0.0334
201
GPA
0.8042 ±0.0041
This paper
236
GAE
actual #236 vs paper #426 190 places worse
0.7978 ±0.0017
best reported
actual #236 vs paper #426 190 places worse
paper 0.7436 ±0.0024
paper 0.7436 ±0.0024 Δ at pub 0.0542
301
JOAO
actual #301 vs paper #317 16 places worse
0.7840 ±0.0050
best reported
actual #301 vs paper #317 16 places worse
paper 0.7807 ±0.0047

Difference within reported uncertainty; not counted as underperformance.

paper 0.7807 ±0.0047 Δ at pub 0.0033
338
InfoGraph
actual #338 vs paper #386 48 places worse
0.7750 ±0.0074
best reported
actual #338 vs paper #386 48 places worse
paper 0.7620 ±0.0106

Difference within reported uncertainty; not counted as underperformance.

paper 0.7620 ±0.0106 Δ at pub 0.0130
409
AD-GCL
actual #409 vs paper #409
0.7518 ±0.0031
actual #409 vs paper #409
paper 0.7518 ±0.0031
paper 0.7518 ±0.0031 Δ at pub 0.0127
#ModelAccuracy ↑Paper
1
WL-MLP
0.9792
2
MSH-GNN
0.9640 ±0.0070
3
CoCN
0.8722 ±0.0013
23
GCN
0.8260 ±0.0220
32
JOAOv2
actual #32 vs paper #240 208 places worse
0.8216 ±0.0017
best reported
actual #32 vs paper #240 208 places worse
paper 0.7553 ±0.0018
paper 0.7553 ±0.0018 Δ at pub 0.0663
80
GraphCL
actual #80 vs paper #260 180 places worse
0.8108 ±0.0017
best reported
actual #80 vs paper #260 180 places worse
paper 0.7423 ±0.0021
paper 0.7423 ±0.0021 Δ at pub 0.0685
91
GAE
actual #91 vs paper #252 161 places worse
0.8078 ±0.0012
best reported
actual #91 vs paper #252 161 places worse
paper 0.7509 ±0.0019
paper 0.7509 ±0.0019 Δ at pub 0.0569
146
InfoGraph
actual #146 vs paper #269 123 places worse
0.7940
best reported
actual #146 vs paper #269 123 places worse
paper 0.7376 ±0.0029
paper 0.7376 ±0.0029 Δ at pub 0.0564
151
MLP
0.7920 ±0.0240
226
GPA
0.7617 ±0.0010
This paper
233
AD-GCL
actual #233 vs paper #233
0.7582 ±0.0026
actual #233 vs paper #233
paper 0.7582 ±0.0026
paper 0.7582 ±0.0026 Δ at pub 0.0252
248
JOAO
actual #248 vs paper #248
0.7535 ±0.0093
best reported
actual #248 vs paper #248
paper 0.7530 ±0.0032

Difference within reported uncertainty; not counted as underperformance.

paper 0.7530 ±0.0032 Δ today 0.0005
#ModelAccuracy ↑Paper
1
OEPG
0.9630 ±0.0090
2
MSH-GNN
0.9580 ±0.0090
3
GNN-AK+
0.9480 ±0.0080
13
GIN
0.9240 ±0.0250
23
GraphCL
actual #23 vs paper #43 20 places worse
0.9169 ±0.0070
best reported
actual #23 vs paper #43 20 places worse
paper 0.8953 ±0.0084
best at pub: Graph Contrastive Learning Automated (ICML 2021)
paper 0.8953 ±0.0084 Δ today 0.0216
24
JOAO
actual #24 vs paper #51 27 places worse
0.8814 ±0.0025
actual #24 vs paper #51 27 places worse
paper 0.8814 ±0.0025
paper 0.8814 ±0.0025 Δ at pub 0.0174 Δ today 0.0352
37
AD-GCL
actual #37 vs paper #37
0.9010 ±0.0015
actual #37 vs paper #37
paper 0.9010 ±0.0015

Improvement within reported uncertainty; not highlighted.

paper 0.9010 ±0.0015 Δ at pub 0.0003
39
GPA
0.8999 ±0.0032
This paper
46
InfoGraph
actual #46 vs paper #49 3 places worse
0.8866 ±0.0095
actual #46 vs paper #49 3 places worse
paper 0.8866 ±0.0095
paper 0.8866 ±0.0095 Δ at pub 0.0616
49
JOAOv2
0.8879 ±0.0065
This paper
55
GAE
actual #55 vs paper #55
0.8769 ±0.0040
actual #55 vs paper #55
paper 0.8769 ±0.0040

Improvement within reported uncertainty; not highlighted.

paper 0.8769 ±0.0040 Δ at pub 0.0059

OGB

#ModelROC-AUC ↑Paper
1
Multi-RF Fusion with Multi-GNN Blending
0.8476 ±0.0002
2
HyperFusion
0.8475 ±0.0003
3
PAS+FPs
0.8420 ±0.0015
9
GatedGCN
0.8090 ±0.0160
109
GIN
0.7780 ±0.0182
250
JOAOv2
actual #250 vs paper #308 58 places worse
0.7440 ±0.0140
best reported
actual #250 vs paper #308 58 places worse
paper 0.5739 ±0.0139
paper 0.5739 ±0.0139 Δ at pub 0.1701
287
GraphCL
actual #287 vs paper #310 23 places worse
0.7060 ±0.0160
best reported
actual #287 vs paper #310 23 places worse
paper 0.5548 ±0.0132
paper 0.5548 ±0.0132 Δ at pub 0.1512
306
GPA
0.6076 ±0.0101
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best at pub — reported value improved over the best known when published Improved at pub — later paper surpassed this result Best today — OK at publication; today's leaderboard best is elsewhere