AAAI 2021

ifMixup: Interpolating Graph Pair to Regularize Graph Classification

Hongyu Guo, Yongyi Mao

Main figure for ifMixup: Interpolating Graph Pair to Regularize Graph Classification

Abstract

We present a simple and yet effective interpolation-based regularization technique, aiming to improve the generalization of Graph Neural Networks (GNNs) on supervised graph classification. We leverage Mixup, an effective regularizer for vision, where random sample pairs and their labels are interpolated to create synthetic images for training. Unlike images with grid-like coordinates, graphs have arbitrary structure and topology, which can be very sensitive to any modification that alters the graph's semantic meanings. This posts two unanswered questions for Mixup-like regularization schemes: Can we directly mix up a pair of graph inputs? If so, how well does such mixing strategy regularize the learning of GNNs? To answer these two questions, we propose ifMixup, which first adds dummy nodes to make two graphs have the same input size and then simultaneously performs linear interpolation between the aligned node feature vectors and the aligned edge representations of the two graphs. We empirically show that such simple mixing schema can effectively regularize the classification learning, resulting in superior predictive accuracy to popular graph augmentation and GNN methods.

Proposed method: ifMixup

Benchmark Results

Tested on 4 datasets 2 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

TU Dortmund

#ModelAccuracy ↑Paper
1
MSH-GNN
0.9910 ±0.0030
2
ID-GNN+BEC
0.9830 ±0.0360
3
SupCosine
0.9830 ±0.0250
39
GIN
0.9330 ±0.0290
116
DropEdge
actual #116 vs paper #437 321 places worse
0.9100 ±0.0570
best reported
actual #116 vs paper #437 321 places worse
paper 0.8550 ±0.0080

Difference within reported uncertainty; not counted as underperformance.

paper 0.8550 ±0.0080 Δ today 0.0550
169
GATv2
actual #169 vs paper #505 336 places worse
0.9040 ±0.0430
best reported
actual #169 vs paper #505 336 places worse
paper 0.8350 ±0.0100
paper 0.8350 ±0.0100 Δ today 0.0690
183
GAT
actual #183 vs paper #518 335 places worse
0.9010 ±0.0580
best reported
actual #183 vs paper #518 335 places worse
paper 0.8310 ±0.0020
paper 0.8310 ±0.0020 Δ at pub 0.0700 Δ today 0.0700
276
ifMixup
0.8900 ±0.0060
This paper
306
GIN Baseline
0.8860 ±0.0110
This paper
316
MixupGraph
0.8840 ±0.0090
This paper
327
Attr. Masking
0.8810 ±0.0040
This paper
363
DropNode
actual #363 vs paper #423 60 places worse
0.8750
best reported
actual #363 vs paper #423 60 places worse
paper 0.8590 ±0.0030
paper 0.8590 ±0.0030 Δ today 0.0160
403
MLP
0.8660 ±0.0495
452
GCN Baseline
0.8500 ±0.0110
This paper
#ModelAccuracy ↑Paper
1
WL-MLP
0.9954
2
GIN
0.9934
3
GCN
0.9727
4
GAT
actual #4 vs paper #300 296 places worse
0.7840 ±0.0040
actual #4 vs paper #300 296 places worse
paper 0.7840 ±0.0040

Improvement within reported uncertainty; not highlighted.

paper 0.7840 ±0.0040 Δ at pub 0.0160 Δ today 0.1783
41
ifMixup
0.8390 ±0.0040
This paper
107
Attr. Masking
0.8250 ±0.0020
This paper
111
DropNode
actual #111 vs paper #191 80 places worse
0.8240
best reported
actual #111 vs paper #191 80 places worse
paper 0.8050 ±0.0190
paper 0.8050 ±0.0190 Δ today 0.0190
117
DropEdge
actual #117 vs paper #176 59 places worse
0.8230 ±0.0010
best reported
actual #117 vs paper #176 59 places worse
paper 0.8110 ±0.0020
paper 0.8110 ±0.0020 Δ today 0.0120
122
MixupGraph
0.8220 ±0.0010
This paper
131
GraphSAGE
0.8200 ±0.0100
143
GIN Baseline
0.8180 ±0.0090
This paper
206
GCN Baseline
0.8040 ±0.0050
This paper
261
GATv2
0.7910 ±0.0140
This paper
#ModelAccuracy ↑Paper
1
MSH-GNN
0.9410 ±0.0330
2
GRDL
0.8260 ±0.0120
3
GMN
0.8225
32
GIN
0.7880 ±0.0410
49
GAT
actual #49 vs paper #298 249 places worse
0.7800 ±0.0300
best reported
actual #49 vs paper #298 249 places worse
paper 0.7430 ±0.0020

Difference within reported uncertainty; not counted as underperformance.

paper 0.7430 ±0.0020 Δ at pub 0.0160 Δ today 0.0370
149
MLP
0.7600 ±0.0320
193
DropEdge
actual #193 vs paper #245 52 places worse
0.7550 ±0.0020
best reported
actual #193 vs paper #245 52 places worse
paper 0.7500 ±0.0030
paper 0.7500 ±0.0030 Δ today 0.0050
216
ifMixup
0.7530 ±0.0080
This paper
234
DropNode
actual #234 vs paper #235 1 place worse
0.7511 ±0.0300
best reported
actual #234 vs paper #235 1 place worse
paper 0.7510 ±0.0050

Difference within reported uncertainty; not counted as underperformance.

paper 0.7510 ±0.0050 Δ today 0.0001
274
Attr. Masking
0.7480 ±0.0050
This paper
292
GIN Baseline
0.7450 ±0.0030
This paper
306
GCN Baseline
0.7420 ±0.0030
This paper
307
MixupGraph
0.7420 ±0.0030
This paper
344
GATv2
0.7350 ±0.0100
This paper
#ModelAccuracy ↑Paper
1
GMN
0.7866
2
MemGNN
0.7550
3
TFGW SP
0.7510 ±0.0500
17
GatedGCN
0.7000 ±0.0494
20
GAT
actual #20 vs paper #167 147 places worse
0.6850 ±0.0524
best reported
actual #20 vs paper #167 147 places worse
paper 0.4680 ±0.0110
best today: GNNBenchmark (JMLR 2023)
paper 0.4680 ±0.0110 Δ at pub 0.0730 Δ today 0.2170
26
GraphSAGE
0.6817 ±0.0545
99
DropEdge
actual #99 vs paper #121 22 places worse
0.5970 ±0.0020
best reported
actual #99 vs paper #121 22 places worse
paper 0.5660 ±0.0060
paper 0.5660 ±0.0060 Δ today 0.0310
108
DropNode
actual #108 vs paper #145 37 places worse
0.5900 ±0.0040
best reported
actual #108 vs paper #145 37 places worse
paper 0.5320 ±0.0060
paper 0.5320 ±0.0060 Δ today 0.0580
121
ifMixup
0.5700 ±0.0140
This paper
124
MLP
0.5583 ±0.0352
132
MixupGraph
0.5510 ±0.0160
This paper
141
GCN Baseline
0.5410 ±0.0010
This paper
149
GIN Baseline
0.5260 ±0.0140
This paper
156
Attr. Masking
0.5060 ±0.0210
This paper
166
GATv2
0.4770 ±0.0210
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best today — OK at publication; today's leaderboard best is elsewhere