LAC: Graph Contrastive Learning with Learnable Augmentation in Continuous Space

Zhenyu Lin, Hongzheng Li, Yingxia Shao, Guanhua Ye, Yawen Li, Quanqing Xu

Main figure for LAC: Graph Contrastive Learning with Learnable Augmentation in Continuous Space

Abstract

Graph Contrastive Learning frameworks have demonstrated success in generating high-quality node representations. The existing research on efficient data augmentation methods and ideal pretext tasks for graph contrastive learning remains limited, resulting in suboptimal node representation in the unsupervised setting. In this paper, we introduce LAC, a graph contrastive learning framework with learnable data augmentation in an orthogonal continuous space. To capture the representative information in the graph data during augmentation, we introduce a continuous view augmenter, that applies both a masked topology augmentation module and a cross-channel feature augmentation module to adaptively augment the topological information and the feature information within an orthogonal continuous space, respectively. The orthogonal nature of continuous space ensures that the augmentation process avoids dimension collapse. To enhance the effectiveness of pretext tasks, we propose an information-theoretic principle named InfoBal and introduce corresponding pretext tasks. These tasks enable the continuous view augmenter to maintain consistency in the representative information across views while maximizing diversity between views, and allow the encoder to fully utilize the representative information in the unsupervised setting. Our experimental results show that LAC significantly outperforms the state-of-the-art frameworks.

Proposed method: LAC

Benchmark Results

Tested on 3 datasets 17 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

#ModelAccuracy ↑Paper
1
LGTL
0.9518 ±0.0011
2
MixHop (LO)
0.9513 ±0.0038
3
GraphSAGE (LO)
0.9511 ±0.0036
18
GCN
0.9296 ±0.0015
136
CCA-SSG
actual #136 vs paper #246 110 places worse
0.8840 ±0.0060
best reported
actual #136 vs paper #246 110 places worse
paper 0.8539 ±0.0051
paper 0.8539 ±0.0051 Δ at pub 0.0301
144
BGRL
actual #144 vs paper #229 85 places worse
0.8830 ±0.0010
best reported
actual #144 vs paper #229 85 places worse
paper 0.8603 ±0.0033
paper 0.8603 ±0.0033 Δ at pub 0.0227
194
GRACE
actual #194 vs paper #242 48 places worse
0.8730 ±0.0010
best reported
actual #194 vs paper #242 48 places worse
paper 0.8551 ±0.0037
paper 0.8551 ±0.0037 Δ at pub 0.0179
196
MVGRL
actual #196 vs paper #270 74 places worse
0.8725 ±0.0012
best reported
actual #196 vs paper #270 74 places worse
paper 0.8427 ±0.0004
204
DGI
actual #204 vs paper #261 57 places worse
0.8708 ±0.0017
best reported
actual #204 vs paper #261 57 places worse
paper 0.8475 ±0.0019
208
COSTA
actual #208 vs paper #230 22 places worse
0.8698 ±0.0033
best reported
actual #208 vs paper #230 22 places worse
paper 0.8601 ±0.0019
210
MLP
0.8690 ±0.0074
229
LAC
0.8605 ±0.0021
This paper
244
ADGCL
0.8545 ±0.0033
This paper
243
GCL-SPAN
actual #243 vs paper #243
0.8545 ±0.0016
actual #243 vs paper #243
paper 0.8545 ±0.0016
paper 0.8545 ±0.0016 Δ at pub 0.0618
251
Node2Vec
actual #251 vs paper #807 556 places worse
0.8520 ±0.0002
best reported
actual #251 vs paper #807 556 places worse
paper 0.7429 ±0.0039
255
GCA
actual #255 vs paper #255
0.8512 ±0.0023
actual #255 vs paper #255
paper 0.8512 ±0.0023
paper 0.8512 ±0.0023 Δ at pub 0.0384
271
AutoGCL
0.8427 ±0.0052
This paper
284
RawFeat
0.8384 ±0.0010
This paper
299
VGAE
actual #299 vs paper #299
0.8307 ±0.0029
actual #299 vs paper #299
paper 0.8307 ±0.0029
paper 0.8307 ±0.0029 Δ at pub 0.0174
304
GAE
actual #304 vs paper #304
0.8294 ±0.0031
actual #304 vs paper #304
paper 0.8294 ±0.0031

Improvement within reported uncertainty; not highlighted.

paper 0.8294 ±0.0031 Δ at pub 0.0105
#ModelAccuracy ↑Paper
1
GraphTARIF
0.9703 ±0.0019
2
GAT
0.9669 ±0.0014
3
GAT+T
0.9660 ±0.0007
7
SAGE
0.9643 ±0.0027
120
LAC
0.9375 ±0.0043
This paper
172
BGRL
actual #172 vs paper #198 26 places worse
0.9337 ±0.0021
best reported
actual #172 vs paper #198 26 places worse
paper 0.9317 ±0.0030

Difference within reported uncertainty; not counted as underperformance.

paper 0.9317 ±0.0030 Δ at pub 0.0020
176
MC-DCD
0.9331 ±0.0013
This paper
186
CCA-SSG
actual #186 vs paper #201 15 places worse
0.9325 ±0.0021
best reported
actual #186 vs paper #201 15 places worse
paper 0.9314 ±0.0014

Difference within reported uncertainty; not counted as underperformance.

paper 0.9314 ±0.0014 Δ at pub 0.0011
191
GCA
actual #191 vs paper #283 92 places worse
0.9320 ±0.0060
best reported
actual #191 vs paper #283 92 places worse
paper 0.9249 ±0.0009
203
GRACE
actual #203 vs paper #324 121 places worse
0.9313 ±0.0017
best reported
actual #203 vs paper #324 121 places worse
paper 0.9215 ±0.0025
paper 0.9215 ±0.0025 Δ at pub 0.0065 Δ today 0.0098
206
MVGRL
actual #206 vs paper #359 153 places worse
0.9310 ±0.0040
best reported
actual #206 vs paper #359 153 places worse
paper 0.9174 ±0.0007
paper 0.9174 ±0.0007 Δ at pub 0.0136
213
DGI
actual #213 vs paper #412 199 places worse
0.9309 ±0.0008
best reported
actual #213 vs paper #412 199 places worse
paper 0.9076 ±0.0029
263
COSTA
actual #263 vs paper #275 12 places worse
0.9267 ±0.0016
best reported
actual #263 vs paper #275 12 places worse
paper 0.9256 ±0.0042

Difference within reported uncertainty; not counted as underperformance.

paper 0.9256 ±0.0042 Δ at pub 0.0011
275
GCL-SPAN
actual #275 vs paper #275
0.9258 ±0.0017
actual #275 vs paper #275
paper 0.9258 ±0.0017
paper 0.9258 ±0.0017 Δ at pub 0.0134
316
AutoGCL
actual #316 vs paper #316
0.9225 ±0.0030
actual #316 vs paper #316
paper 0.9225 ±0.0030

Improvement within reported uncertainty; not highlighted.

paper 0.9225 ±0.0030 Δ at pub 0.0027
317
VGAE
actual #317 vs paper #324 7 places worse
0.9224
best reported
actual #317 vs paper #324 7 places worse
paper 0.9220 ±0.0011

Difference within reported uncertainty; not counted as underperformance.

paper 0.9220 ±0.0011 Δ at pub 0.0004
352
GAE
actual #352 vs paper #366 14 places worse
0.9185 ±0.0006
best reported
actual #352 vs paper #366 14 places worse
paper 0.9162 ±0.0013
paper 0.9162 ±0.0013 Δ at pub 0.0023
376
ADGCL
0.9145 ±0.0012
This paper
384
JOAO-v2
actual #384 vs paper #384
0.9139 ±0.0025
actual #384 vs paper #384
paper 0.9139 ±0.0025

Improvement within reported uncertainty; not highlighted.

paper 0.9139 ±0.0025 Δ at pub 0.0024
390
MLP
0.9128 ±0.0063
457
Node2Vec
actual #457 vs paper #459 2 places worse
0.8970 ±0.0010
best reported
actual #457 vs paper #459 2 places worse
paper 0.8961 ±0.0024

Difference within reported uncertainty; not counted as underperformance.

paper 0.8961 ±0.0024 Δ at pub 0.0009 Δ today 0.0009
522
RawFeat
0.8352 ±0.0031
This paper
#ModelAccuracy ↑Paper
1
GraphTARIF
0.9461 ±0.0017
2
GCN
0.9412 ±0.0008
3
GAT
0.9398 ±0.0022
8
SAGE
0.9351 ±0.0006
70
LAC
0.9055 ±0.0016
This paper
79
BGRL
actual #79 vs paper #183 104 places worse
0.9034 ±0.0019
best reported
actual #79 vs paper #183 104 places worse
paper 0.8854 ±0.0003
paper 0.8854 ±0.0003 Δ at pub 0.0180 Δ today 0.0180
105
GCL-SPAN
0.8996 ±0.0013
This paper
139
GRACE
actual #139 vs paper #245 106 places worse
0.8953 ±0.0035
best reported
actual #139 vs paper #245 106 places worse
paper 0.8725 ±0.0025
paper 0.8725 ±0.0025 Δ at pub 0.0228 Δ today 0.0228
159
CCA-SSG
actual #159 vs paper #171 12 places worse
0.8896 ±0.0013
best reported
actual #159 vs paper #171 12 places worse
paper 0.8874 ±0.0028

Difference within reported uncertainty; not counted as underperformance.

paper 0.8874 ±0.0028 Δ today 0.0022
160
GCA
actual #160 vs paper #213 53 places worse
0.8894 ±0.0015
best reported
actual #160 vs paper #213 53 places worse
paper 0.8785 ±0.0031
paper 0.8785 ±0.0031 Δ at pub 0.0109 Δ today 0.0109
171
MC-DCD
0.8878 ±0.0025
This paper
179
MVGRL
actual #179 vs paper #235 56 places worse
0.8861 ±0.0064
best reported
actual #179 vs paper #235 56 places worse
paper 0.8752 ±0.0011
192
COSTA
actual #192 vs paper #193 1 place worse
0.8837 ±0.0022
best reported
actual #192 vs paper #193 1 place worse
paper 0.8832 ±0.0030

Difference within reported uncertainty; not counted as underperformance.

paper 0.8832 ±0.0030 Δ at pub 0.0005
210
DGI
actual #210 vs paper #354 144 places worse
0.8798 ±0.0081
best reported
actual #210 vs paper #354 144 places worse
paper 0.8395 ±0.0047
paper 0.8395 ±0.0047 Δ at pub 0.0403
246
VGAE
actual #246 vs paper #274 28 places worse
0.8721 ±0.0026
best reported
actual #246 vs paper #274 28 places worse
paper 0.8637 ±0.0021
paper 0.8637 ±0.0021 Δ at pub 0.0084
248
AutoGCL
0.8711 ±0.0084
This paper
261
JOAO-v2
0.8673 ±0.0044
This paper
264
GAE
actual #264 vs paper #305 41 places worse
0.8666 ±0.0007
best reported
actual #264 vs paper #305 41 places worse
paper 0.8527 ±0.0019
paper 0.8527 ±0.0019 Δ at pub 0.0139
287
ADGCL
0.8603 ±0.0021
This paper
320
MLP
0.8463
330
Node2Vec
actual #330 vs paper #334 4 places worse
0.8440 ±0.0010
best reported
actual #330 vs paper #334 4 places worse
paper 0.8439 ±0.0008

Difference within reported uncertainty; not counted as underperformance.

paper 0.8439 ±0.0008 Δ at pub 0.0001 Δ today 0.0001
438
RawFeat
0.7853
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best at pub — reported value improved over the best known when published Best today — OK at publication; today's leaderboard best is elsewhere