Supervised Contrastive Learning with Structure Inference for Graph Classification

Junzhong Ji, Hao Jia, Yating Ren, Minglong Lei

Main figure for Supervised Contrastive Learning with Structure Inference for Graph Classification

Abstract

Advanced graph neural networks have shown great potentials in graph classification tasks recently. Different from node classification where node embeddings aggregated from local neighbors can be directly used to learn node labels, graph classification requires a hierarchical accumulation of different levels of topological information to generate discriminative graph embeddings. Still, how to fully explore graph structures and formulate an effective graph classification pipeline remains rudimentary. In this paper, we propose a novel graph neural network based on supervised contrastive learning with structure inference for graph classification. First, we propose a data-driven graph augmentation strategy that can discover additional connections to enhance the existing edge set. Concretely, we resort to a structure inference stage based on diffusion cascades to recover possible connections with high node similarities. Second, to improve the contrastive power of graph neural networks, we propose to use a supervised contrastive loss for graph classification. With the integration of label information, the one-vs-many contrastive learning can be extended to a many-vs-many setting, so that the graph-level embeddings with higher topological similarities will be pulled closer. The supervised contrastive loss and structure inference can be naturally incorporated within the hierarchical graph neural networks where the topological patterns can be fully explored to produce discriminative graph embeddings. Experiment results show the effectiveness of the proposed method compared with recent state-of-the-art methods.

Proposed method: SupCosine

Benchmark Results

Tested on 1 dataset 1 baseline value reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

TU Dortmund

#ModelAccuracy ↑Paper
1
MSH-GNN
0.9910 ±0.0030
3
SupCosine
0.9830 ±0.0250
This paper
2
ID-GNN+BEC
0.9830 ±0.0360
7
SUGAR
actual #7 vs paper #7
0.9674 ±0.0455
best reported
actual #7 vs paper #7
paper 0.9670 ±0.0460

Difference within reported uncertainty; not counted as underperformance.

paper 0.9670 ±0.0460 Δ at pub 0.0004
28
sGIN
0.9410 ±0.0270
This paper
39
GIN
actual #39 vs paper #237 198 places worse
0.9330 ±0.0290
best reported
actual #39 vs paper #237 198 places worse
paper 0.8940 ±0.0560

Difference within reported uncertainty; not counted as underperformance.

paper 0.8940 ±0.0560 Δ at pub 0.0390
63
GCN
0.9220 ±0.0440
140
PPGN
0.9060 ±0.0870
This paper
159
WL
actual #159 vs paper #498 339 places worse
0.9040 ±0.0570
best reported
actual #159 vs paper #498 339 places worse
paper 0.8380 ±0.0150
paper 0.8380 ±0.0150 Δ at pub 0.0240 Δ today 0.0660
158
GraphSAGE
actual #158 vs paper #623 465 places worse
0.9040 ±0.0780
best reported
actual #158 vs paper #623 465 places worse
paper 0.7980 ±0.1390

Difference within reported uncertainty; not counted as underperformance.

paper 0.7980 ±0.1390 Δ at pub 0.0590 Δ today 0.1060
175
RetGK
0.9030 ±0.0110
This paper
202
GMT
actual #202 vs paper #511 309 places worse
0.8990 ±0.0420
best reported
actual #202 vs paper #511 309 places worse
paper 0.8340 ±0.0130
paper 0.8340 ±0.0130 Δ today 0.0650
216
M-GCL
0.8970 ±0.0110
This paper
230
SGN
0.8950 ±0.0740
This paper
257
InfoGraph
actual #257 vs paper #272 15 places worse
0.8930 ±0.0514
best reported
actual #257 vs paper #272 15 places worse
paper 0.8900 ±0.0110
best at pub: Graph Contrastive Learning Automated (ICML 2021)

Difference within reported uncertainty; not counted as underperformance.

paper 0.8900 ±0.0110 Δ at pub 0.0001 Δ today 0.0030
301
CapsGNN
actual #301 vs paper #400 99 places worse
0.8867 ±0.0688
best reported
actual #301 vs paper #400 99 places worse
paper 0.8670 ±0.0690

Difference within reported uncertainty; not counted as underperformance.

paper 0.8670 ±0.0690 Δ today 0.0197
335
ndp
0.8790 ±0.0570
This paper
354
GraphCL
actual #354 vs paper #396 42 places worse
0.8766 ±0.0103
best reported
actual #354 vs paper #396 42 places worse
paper 0.8680 ±0.0130

Difference within reported uncertainty; not counted as underperformance.

paper 0.8680 ±0.0130 Δ at pub 0.0086
365
DGK
actual #365 vs paper #366 1 place worse
0.8744 ±0.0272
best reported
actual #365 vs paper #366 1 place worse
paper 0.8740 ±0.0270
best at pub: Provably Powerful Graph Networks (NeurIPS 2019)

Difference within reported uncertainty; not counted as underperformance.

paper 0.8740 ±0.0270 Δ at pub 0.0004 Δ today 0.0004
382
DAGCN
0.8720 ±0.0610
This paper
403
MLP
0.8660 ±0.0495
419
GXN
actual #419 vs paper #419
0.8610 ±0.0830
actual #419 vs paper #419
paper 0.8610 ±0.0830

Improvement within reported uncertainty; not highlighted.

paper 0.8610 ±0.0830 Δ at pub 0.1200
506
GK
actual #506 vs paper #506
0.8350 ±0.0060
actual #506 vs paper #506
paper 0.8350 ±0.0060

Improvement within reported uncertainty; not highlighted.

paper 0.8350 ±0.0060 Δ at pub 0.0180
623
SAGPool
actual #623 vs paper #684 61 places worse
0.7986 ±0.0236
best reported
actual #623 vs paper #684 61 places worse
paper 0.7680 ±0.0210

Difference within reported uncertainty; not counted as underperformance.

paper 0.7680 ±0.0210 Δ at pub 0.0170 Δ today 0.0306
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best today — OK at publication; today's leaderboard best is elsewhere