NeurIPS 2021

AutoGEL: An Automated Graph Neural Network with Explicit Link Information

Zhiling Wang, Shimin Di, Lei Chen

Main figure for AutoGEL: An Automated Graph Neural Network with Explicit Link Information

Abstract

Recently, Graph Neural Networks (GNNs) have gained popularity in a variety of real-world scenarios. Despite the great success, the architecture design of GNNs heavily relies on manual labor. Thus, automated graph neural network (AutoGNN) has attracted interest and attention from the research community, which makes significant performance improvements in recent years. However, existing AutoGNN works mainly adopt an implicit way to model and leverage the link information in the graphs, which is not well regularized to the link prediction task on graphs, and limits the performance of AutoGNN for other graph tasks. In this paper, we present a novel AutoGNN work that explicitly models the link information, abbreviated to AutoGEL. In such a way, AutoGEL can handle the link prediction task and improve the performance of AutoGNNs on the node classification and graph classification task. Specifically, AutoGEL proposes a novel search space containing various design dimensions at both intra-layer and inter-layer designs and adopts a more robust differentiable search algorithm to further improve efficiency and effectiveness. Experimental results on benchmark data sets demonstrate the superiority of AutoGEL on several tasks.

Proposed method: AutoGEL

Benchmark Results

Tested on 4 datasets 3 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

TU Dortmund

#ModelAccuracy ↑Paper
1
MSH-GNN
0.9910 ±0.0030
2
ID-GNN+BEC
0.9830 ±0.0360
3
SupCosine
0.9830 ±0.0250
39
GIN
actual #39 vs paper #237 198 places worse
0.9330 ±0.0290
best reported
actual #39 vs paper #237 198 places worse
paper 0.8940

Difference within reported uncertainty; not counted as underperformance.

paper 0.8940 Δ at pub 0.0390
54
PatchySan
actual #54 vs paper #799 745 places worse
0.9260 ±0.0420
best reported
actual #54 vs paper #799 745 places worse
paper 0.4520
paper 0.4520 Δ at pub 0.4740 Δ today 0.4740
63
GCN
actual #63 vs paper #432 369 places worse
0.9220 ±0.0440
best reported
actual #63 vs paper #432 369 places worse
paper 0.8560

Difference within reported uncertainty; not counted as underperformance.

paper 0.8560 Δ at pub 0.0350 Δ today 0.0660
158
GraphSAGE
actual #158 vs paper #446 288 places worse
0.9040 ±0.0780
best reported
actual #158 vs paper #446 288 places worse
paper 0.8510

Difference within reported uncertainty; not counted as underperformance.

paper 0.8510 Δ at pub 0.0060 Δ today 0.0530
403
MLP
0.8660 ±0.0495
425
DGCNN
actual #425 vs paper #425
0.8583 ±0.0116
best reported
actual #425 vs paper #425
paper 0.8580
718
SAGE
0.7490 ±0.0870
799
NAS-GCN
0.4780
This paper
#ModelAccuracy ↑Paper
1
GraphSNN
0.7790 ±0.0360
2
SIN
0.7560 ±0.0320
3
CIN
0.7560 ±0.0370
5
GIN
actual #5 vs paper #31 26 places worse
0.7510 ±0.0510
best reported
actual #5 vs paper #31 26 places worse
paper 0.5230
best at pub: Graph Homomorphism Convolution (ICML 2020)
paper 0.5230 Δ today 0.2280
38
GCN
0.5190
This paper
46
GraphSAGE
0.5090
This paper
78
DGCNN
actual #78 vs paper #90 12 places worse
0.4900 ±0.0356
best reported
actual #78 vs paper #90 12 places worse
paper 0.4780

Difference within reported uncertainty; not counted as underperformance.

paper 0.4780 Δ at pub 0.0120
86
SAGE
0.4853 ±0.0543
91
NAS-GCN
0.4780
This paper
106
PatchySan
0.4520
This paper

Knowledge Graphs

#ModelMRR ↑Paper
1
FedSage
0.8426 ±0.0026
2
FED-PUB
0.8348 ±0.0025
3
FedAvg
0.8252 ±0.0023
7
GCN
0.8171 ±0.0030
61
AutoGEL
0.4790
This paper
60
CompGCN
0.4790
This paper
62
RotatE
actual #62 vs paper #62
0.4770
best reported
actual #62 vs paper #62
paper 0.4760
paper 0.4760 Δ at pub 0.0010 Δ today 0.0010
63
ConvR
0.4750
This paper
69
SACN
0.4700
This paper
73
HyperER
0.4650
This paper
84
ComplEx
0.4400
This paper
88
DisMult
0.4300
This paper
89
ConvE
0.4300
This paper
98
ConvKB
actual #98 vs paper #98
0.2650
best reported
actual #98 vs paper #98
paper 0.2490
100
TransE
actual #100 vs paper #101 1 place worse
0.2310
best reported
actual #100 vs paper #101 1 place worse
paper 0.2260
#ModelMRR ↑Paper
1
sum, Msg_r^3, f(t)=0
0.8500
2
sum, Msg_r^3, f(t)=t
0.8440
3
PNA, Msg_r^1, f(t)=0
0.8010
34
AutoGEL
0.3570
This paper
35
CompGCN
0.3550
This paper
44
ConvR
0.3500
This paper
45
SACN
0.3500
This paper
47
ComplEx
actual #47 vs paper #91 44 places worse
0.3460
best reported
actual #47 vs paper #91 44 places worse
paper 0.2470
best at pub: Knowledge Graphs (AAAI 2017)
paper 0.2470 Δ today 0.0990
57
HyperER
0.3410
This paper
58
RotatE
0.3380
This paper
67
TransE
actual #67 vs paper #79 12 places worse
0.3300
best reported
actual #67 vs paper #79 12 places worse
paper 0.2940
71
ConvE
0.3250
This paper
83
ConvKB
actual #83 vs paper #91 8 places worse
0.2890
best reported
actual #83 vs paper #91 8 places worse
paper 0.2430
89
R-GCN
actual #89 vs paper #90 1 place worse
0.2490
best reported
actual #89 vs paper #90 1 place worse
paper 0.2480
paper 0.2480 Δ at pub 0.0010
91
VR-GCN
0.2480
This paper
92
DisMult
actual #92 vs paper #92
0.2430
best reported
actual #92 vs paper #92
paper 0.2410
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best today — OK at publication; today's leaderboard best is elsewhere