ICLR 2022

Automated Data Augmentations for Graph Classification

Youzhi Luo, Michael McThrow, Wing Yee Au, Tao Komikado, Kanji Uchino, Koji Maruhashi, Shuiwang Ji

Main figure for Automated Data Augmentations for Graph Classification

Abstract

Data augmentations are effective in improving the invariance of learning machines. We argue that the core challenge of data augmentations lies in designing data transformations that preserve labels. This is relatively straightforward for images, but much more challenging for graphs. In this work, we propose GraphAug, a novel automated data augmentation method aiming at computing label-invariant augmentations for graph classification. Instead of using uniform transformations as in existing studies, GraphAug uses an automated augmentation model to avoid compromising critical label-related information of the graph, thereby producing label-invariant augmentations at most times. To ensure label-invariance, we develop a training method based on reinforcement learning to maximize an estimated label-invariance probability. Experiments show that GraphAug outperforms previous graph augmentation methods on various graph classification tasks.

Proposed method: GraphAug

Benchmark Results

Tested on 6 datasets 9 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

TU Dortmund

#ModelAccuracy ↑Paper
1
MSH-GNN
0.9910 ±0.0030
2
ID-GNN+BEC
0.9830 ±0.0360
3
SupCosine
0.9830 ±0.0250
39
GIN
0.9330 ±0.0290
116
DropEdge
actual #116 vs paper #665 549 places worse
0.9100 ±0.0570
best reported
actual #116 vs paper #665 549 places worse
paper 0.7790 ±0.0050
paper 0.7790 ±0.0050 Δ at pub 0.0760 Δ today 0.1310
178
AD-GCL
actual #178 vs paper #499 321 places worse
0.9028 ±0.0401
best reported
actual #178 vs paper #499 321 places worse
paper 0.8370 ±0.0100
paper 0.8370 ±0.0100 Δ at pub 0.0600 Δ today 0.0658
222
JOAOv2
actual #222 vs paper #673 451 places worse
0.8967 ±0.0524
best reported
actual #222 vs paper #673 451 places worse
paper 0.7750 ±0.0160
best at pub: Graph Contrastive Learning Automated (ICML 2021)
paper 0.7750 ±0.0160 Δ at pub 0.1017 Δ today 0.1217
302
AutoGCL
actual #302 vs paper #656 354 places worse
0.8864 ±0.0108
best reported
actual #302 vs paper #656 354 places worse
paper 0.7830 ±0.0220
paper 0.7830 ±0.0220 Δ today 0.1034
403
MLP
0.8660 ±0.0495
442
GraphAug
0.8530 ±0.0080
This paper
510
FLAG
0.8350 ±0.0150
This paper
538
No augmentation
0.8270 ±0.0130
This paper
600
G-Mixup
0.8050 ±0.0200
This paper
626
M-Mixup
0.7980 ±0.0150
This paper
651
Uniform MaskNF
0.7880 ±0.0120
This paper
652
Uniform DropNode
0.7870 ±0.0030
This paper
666
Uniform Mixture
0.7790 ±0.0140
This paper
688
Uniform PerturbEdge
0.7640 ±0.0080
This paper
#ModelAccuracy ↑Paper
1
WL-MLP
0.9954
2
GIN
0.9934
3
GCN
0.9727
117
DropEdge
actual #117 vs paper #318 201 places worse
0.8230 ±0.0010
best reported
actual #117 vs paper #318 201 places worse
paper 0.7800 ±0.0020
paper 0.7800 ±0.0020 Δ at pub 0.0310 Δ today 0.0430
133
AutoGCL
actual #133 vs paper #440 307 places worse
0.8200 ±0.0029
best reported
actual #133 vs paper #440 307 places worse
paper 0.7370 ±0.0020
paper 0.7370 ±0.0020 Δ today 0.0830
131
GraphSAGE
0.8200 ±0.0100
146
JOAOv2
actual #146 vs paper #510 364 places worse
0.8170 ±0.0026
best reported
actual #146 vs paper #510 364 places worse
paper 0.6700 ±0.0060
best at pub: Graph Contrastive Learning Automated (ICML 2021)
paper 0.6700 ±0.0060 Δ at pub 0.1136 Δ today 0.1470
148
GraphAug
0.8160 ±0.0010
This paper
205
No augmentation
0.8040 ±0.0030
This paper
204
FLAG
0.8040 ±0.0020
This paper
213
M-Mixup
0.8030 ±0.0030
This paper
252
Uniform MaskNF
0.7940 ±0.0020
This paper
280
Uniform DropNode
0.7870 ±0.0030
This paper
306
Uniform Mixture
0.7830 ±0.0030
This paper
312
G-Mixup
actual #312 vs paper #502 190 places worse
0.7820 ±0.0158
best reported
actual #312 vs paper #502 190 places worse
paper 0.6860 ±0.0370
best today: Graph Mixup with Soft Alignments (ICML 2023)
paper 0.6860 ±0.0370 Δ today 0.0960
409
AD-GCL
actual #409 vs paper #533 124 places worse
0.7518 ±0.0031
best reported
actual #409 vs paper #533 124 places worse
paper 0.6410 ±0.0040
paper 0.6410 ±0.0040 Δ today 0.1108
573
Uniform PerturbEdge
0.5450 ±0.0060
This paper
#ModelAccuracy ↑Paper
1
MSH-GNN
0.9410 ±0.0330
2
GRDL
0.8260 ±0.0120
3
GMN
0.8225
32
GIN
0.7880 ±0.0410
149
MLP
0.7600 ±0.0320
168
AutoGCL
actual #168 vs paper #468 300 places worse
0.7580 ±0.0036
best reported
actual #168 vs paper #468 300 places worse
paper 0.6840 ±0.0080
paper 0.6840 ±0.0080 Δ today 0.0740
174
G-Mixup
actual #174 vs paper #429 255 places worse
0.7574 ±0.0312
best reported
actual #174 vs paper #429 255 places worse
paper 0.7150 ±0.0060
paper 0.7150 ±0.0060 Δ today 0.0424
180
M-Mixup
actual #180 vs paper #447 267 places worse
0.7565 ±0.0451
best reported
actual #180 vs paper #447 267 places worse
paper 0.7060 ±0.0030
paper 0.7060 ±0.0030 Δ today 0.0505
193
DropEdge
actual #193 vs paper #445 252 places worse
0.7550 ±0.0020
best reported
actual #193 vs paper #445 252 places worse
paper 0.7070 ±0.0020
paper 0.7070 ±0.0020 Δ at pub 0.0430 Δ today 0.0480
297
FLAG
actual #297 vs paper #443 146 places worse
0.7431 ±0.0421
best reported
actual #297 vs paper #443 146 places worse
paper 0.7090 ±0.0070
paper 0.7090 ±0.0070 Δ today 0.0341
303
GraphAug
actual #303 vs paper #402 99 places worse
0.7421 ±0.0370
best reported
actual #303 vs paper #402 99 places worse
paper 0.7220 ±0.0040

Difference within reported uncertainty; not counted as underperformance.

paper 0.7220 ±0.0040 Δ today 0.0201
309
JOAOv2
actual #309 vs paper #459 150 places worse
0.7407 ±0.0110
best reported
actual #309 vs paper #459 150 places worse
paper 0.7000 ±0.0030
paper 0.7000 ±0.0030 Δ at pub 0.0407 Δ today 0.0407
338
AD-GCL
actual #338 vs paper #459 121 places worse
0.7360 ±0.0070
best reported
actual #338 vs paper #459 121 places worse
paper 0.6990 ±0.0080
446
Uniform DropNode
0.7070 ±0.0040
This paper
447
Uniform Mixture
0.7070 ±0.0040
This paper
454
No augmentation
0.7040 ±0.0040
This paper
456
Uniform MaskNF
0.7020 ±0.0080
This paper
476
Uniform PerturbEdge
0.6680 ±0.0060
This paper
#ModelAccuracy ↑Paper
1
WL-MLP
0.9792
2
MSH-GNN
0.9640 ±0.0070
3
CoCN
0.8722 ±0.0013
19
GraphAug
0.8290 ±0.0020
This paper
23
GCN
0.8260 ±0.0220
32
JOAOv2
actual #32 vs paper #307 275 places worse
0.8216 ±0.0017
best reported
actual #32 vs paper #307 275 places worse
paper 0.6880 ±0.0030
paper 0.6880 ±0.0030 Δ at pub 0.0673 Δ today 0.1336
46
Uniform PerturbEdge
0.8160 ±0.0030
This paper
51
Uniform MaskNF
0.8150 ±0.0020
This paper
52
Uniform DropNode
0.8150 ±0.0040
This paper
53
Uniform Mixture
0.8150 ±0.0030
This paper
74
DropEdge
0.8120 ±0.0030
This paper
77
M-Mixup
0.8110 ±0.0050
This paper
78
G-Mixup
0.8110 ±0.0090
This paper
99
No augmentation
0.8060 ±0.0030
This paper
110
FLAG
0.8030 ±0.0060
This paper
151
MLP
0.7920 ±0.0240
204
AutoGCL
actual #204 vs paper #256 52 places worse
0.7716 ±0.0148
best reported
actual #204 vs paper #256 52 places worse
paper 0.7450 ±0.0020
paper 0.7450 ±0.0020 Δ today 0.0266
233
AD-GCL
actual #233 vs paper #314 81 places worse
0.7582 ±0.0026
best reported
actual #233 vs paper #314 81 places worse
paper 0.6700 ±0.0080
paper 0.6700 ±0.0080 Δ at pub 0.0630 Δ today 0.0882
#ModelAccuracy ↑Paper
1
ID-GNN+BEC
0.8150 ±0.0240
2
GIN+BEC
0.8080 ±0.0330
3
GCN+BEC
0.7990 ±0.0340
4
GCN
0.7980 ±0.0230
21
GraphAug
0.7620 ±0.0040
This paper
43
G-Mixup
0.7480 ±0.0040
This paper
44
FLAG
0.7470 ±0.0080
This paper
67
M-Mixup
0.7360 ±0.0040
This paper
74
DropEdge
0.7330 ±0.0120
This paper
73
AutoGCL
actual #73 vs paper #132 59 places worse
0.7330 ±0.0040
best reported
actual #73 vs paper #132 59 places worse
paper 0.7070 ±0.0070
paper 0.7070 ±0.0070 Δ today 0.0260
80
No augmentation
0.7310 ±0.0040
This paper
83
Uniform Mixture
0.7300 ±0.0090
This paper
91
Uniform DropNode
0.7280 ±0.0060
This paper
92
Uniform PerturbEdge
0.7280 ±0.0070
This paper
109
Uniform MaskNF
0.7200 ±0.0060
This paper
115
JOAOv2
actual #115 vs paper #132 17 places worse
0.7160 ±0.0086
best reported
actual #115 vs paper #132 17 places worse
paper 0.7070 ±0.0080

Difference within reported uncertainty; not counted as underperformance.

paper 0.7070 ±0.0080 Δ today 0.0090
119
AD-GCL
actual #119 vs paper #123 4 places worse
0.7150 ±0.0100
best reported
actual #119 vs paper #123 4 places worse
paper 0.7120 ±0.0080

Difference within reported uncertainty; not counted as underperformance.

paper 0.7120 ±0.0080 Δ today 0.0030

OGB

#ModelROC-AUC ↑Paper
1
Multi-RF Fusion with Multi-GNN Blending
0.8476 ±0.0002
2
HyperFusion
0.8475 ±0.0003
3
PAS+FPs
0.8420 ±0.0015
9
GatedGCN
0.8090 ±0.0160
109
GIN
0.7780 ±0.0182
127
M-Mixup
actual #127 vs paper #229 102 places worse
0.7742 ±0.0077
best reported
actual #127 vs paper #229 102 places worse
paper 0.7530 ±0.0130
best today: Graph Mixup with Soft Alignments (ICML 2023)
paper 0.7530 ±0.0130 Δ today 0.0212
128
G-Mixup
actual #128 vs paper #145 17 places worse
0.7742 ±0.0145
best reported
actual #128 vs paper #145 17 places worse
paper 0.7710 ±0.0050
best today: Graph Mixup with Soft Alignments (ICML 2023)

Difference within reported uncertainty; not counted as underperformance.

paper 0.7710 ±0.0050 Δ today 0.0032
129
GraphAug
0.7740 ±0.0100
This paper
177
FLAG
0.7650 ±0.0110
This paper
188
DropEdge
0.7620 ±0.0100
This paper
189
AD-GCL
0.7620 ±0.0130
This paper
219
No augmentation
0.7560 ±0.0140
This paper
225
Uniform PerturbEdge
0.7550 ±0.0130
This paper
247
Uniform Mixture
0.7460 ±0.0100
This paper
250
JOAOv2
0.7440 ±0.0140
This paper
257
Uniform MaskNF
0.7410 ±0.0100
This paper
285
Uniform DropNode
0.7170 ±0.0110
This paper
288
AutoGCL
0.7040 ±0.0160
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best today — OK at publication; today's leaderboard best is elsewhere