Learning Graph Representation by Aggregating Subgraphs via Mutual Information Maximization

Ziwen Liu, Chenguang Wang, Congying Han, Tiande Guo

Main figure for Learning Graph Representation by Aggregating Subgraphs via Mutual Information Maximization

Abstract

In this paper, we introduce a self-supervised learning method to enhance the graph-level representations with the help of a set of subgraphs. For this purpose, we propose a universal framework to generate subgraphs in an auto-regressive way and then using these subgraphs to guide the learning of graph representation by Graph Neural Networks. Under this framework, we can get a comprehensive understanding of the graph structure in a learnable way. And to fully capture enough information of original graphs, we design three information aggregators: attribute-conv, layer-conv and subgraph-conv to gather information from different aspects. And to achieve efficient and effective contrastive learning, a Head-Tail contrastive construction is proposed to provide abundant negative samples. Under all proposed components which can be generalized to any Graph Neural Networks, in the unsupervised case, we achieve new state-of-the-art results in several benchmarks. We also evaluate our model on semi-supervised learning tasks and make a fair comparison to state-of-the-art semi-supervised methods.

Proposed method: GDN

Benchmark Results

Tested on 1 dataset 1 baseline value reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

TU Dortmund

#ModelAccuracy ↑Paper
1
MSH-GNN
0.9910 ±0.0030
2
ID-GNN+BEC
0.9830 ±0.0360
3
SupCosine
0.9830 ±0.0250
39
GIN
0.9330 ±0.0290
63
GCN
actual #63 vs paper #432 369 places worse
0.9220 ±0.0440
best reported
actual #63 vs paper #432 369 places worse
paper 0.8560 ±0.0580
best at pub: Hierarchical Graph Capsule Network (AAAI 2020)

Difference within reported uncertainty; not counted as underperformance.

paper 0.8560 ±0.0580 Δ at pub 0.0160 Δ today 0.0660
78
GDN
0.9180 ±0.0050
This paper
159
WL
actual #159 vs paper #597 438 places worse
0.9040 ±0.0570
best reported
actual #159 vs paper #597 438 places worse
paper 0.8070 ±0.0300
paper 0.8070 ±0.0300 Δ at pub 0.0550 Δ today 0.0970
158
GraphSAGE
actual #158 vs paper #587 429 places worse
0.9040 ±0.0780
best reported
actual #158 vs paper #587 429 places worse
paper 0.8110 ±0.0760

Difference within reported uncertainty; not counted as underperformance.

paper 0.8110 ±0.0760 Δ at pub 0.0460 Δ today 0.0930
183
GAT
actual #183 vs paper #237 54 places worse
0.9010 ±0.0580
best reported
actual #183 vs paper #237 54 places worse
paper 0.8940 ±0.0610

Difference within reported uncertainty; not counted as underperformance.

paper 0.8940 ±0.0610 Δ today 0.0070
238
GIN-0
0.8940 ±0.0560
This paper
257
InfoGraph
actual #257 vs paper #272 15 places worse
0.8930 ±0.0514
best reported
actual #257 vs paper #272 15 places worse
paper 0.8900 ±0.0110

Difference within reported uncertainty; not counted as underperformance.

paper 0.8900 ±0.0110 Δ today 0.0030
268
hassani2020contrastive
0.8920 ±0.0110
This paper
274
GIN-ε
0.8900 ±0.0600
This paper
332
MLG
actual #332 vs paper #334 2 places worse
0.8794 ±0.0161
best reported
actual #332 vs paper #334 2 places worse
paper 0.8790 ±0.0160
best at pub: The Multiscale Laplacian Graph Kernel (NeurIPS 2016)

Difference within reported uncertainty; not counted as underperformance.

paper 0.8790 ±0.0160 Δ at pub 0.0004
365
DGK
actual #365 vs paper #366 1 place worse
0.8744 ±0.0272
best reported
actual #365 vs paper #366 1 place worse
paper 0.8740 ±0.0270
best at pub: Provably Powerful Graph Networks (NeurIPS 2019)

Difference within reported uncertainty; not counted as underperformance.

paper 0.8740 ±0.0270 Δ at pub 0.0004 Δ today 0.0004
403
MLP
0.8660 ±0.0495
407
graph2vec
actual #407 vs paper #515 108 places worse
0.8644 ±0.0125
best reported
actual #407 vs paper #515 108 places worse
paper 0.8320 ±0.0960

Difference within reported uncertainty; not counted as underperformance.

paper 0.8320 ±0.0960 Δ at pub 0.0048 Δ today 0.0324
426
SP
actual #426 vs paper #443 17 places worse
0.8580 ±0.0020
best reported
actual #426 vs paper #443 17 places worse
paper 0.8520 ±0.0240
best at pub: Propagation Kernels (2014)

Difference within reported uncertainty; not counted as underperformance.

paper 0.8520 ±0.0240 Δ at pub 0.0060
472
Sub2Vec
actual #472 vs paper #783 311 places worse
0.6160 ±0.1580
actual #472 vs paper #783 311 places worse
paper 0.6160 ±0.1580

Improvement within reported uncertainty; not highlighted.

paper 0.6160 ±0.1580 Δ at pub 0.0050
501
RW
actual #501 vs paper #501
0.8370 ±0.0150
actual #501 vs paper #501
paper 0.8370 ±0.0150

Improvement within reported uncertainty; not highlighted.

paper 0.8370 ±0.0150 Δ at pub 0.0002
506
GK
actual #506 vs paper #575 69 places worse
0.8350 ±0.0060
best reported
actual #506 vs paper #575 69 places worse
paper 0.8170 ±0.0210

Difference within reported uncertainty; not counted as underperformance.

paper 0.8170 ±0.0210 Δ today 0.0180
746
Node2Vec
actual #746 vs paper #746
0.7263 ±0.1020
best reported
actual #746 vs paper #746
paper 0.7260 ±0.1020

Difference within reported uncertainty; not counted as underperformance.

paper 0.7260 ±0.1020 Δ today 0.0003
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green)