NeurIPS 2022

Neo-GNNs: Neighborhood Overlap-aware Graph Neural Networks for Link Prediction

Seongjun Yun, Seoyoon Kim, Junhyun Lee, Jaewoo Kang, Hyunwoo J. Kim

Main figure for Neo-GNNs: Neighborhood Overlap-aware Graph Neural Networks for Link Prediction

Abstract

Graph Neural Networks (GNNs) have been widely applied to various fields for learning over graph-structured data. They have shown significant improvements over traditional heuristic methods in various tasks such as node classification and graph classification. However, since GNNs heavily rely on smoothed node features rather than graph structure, they often show poor performance than simple heuristic methods in link prediction where the structural information, e.g., overlapped neighborhoods, degrees, and shortest paths, is crucial. To address this limitation, we propose Neighborhood Overlap-aware Graph Neural Networks (Neo-GNNs) that learn useful structural features from an adjacency matrix and estimate overlapped neighborhoods for link prediction. Our Neo-GNNs generalize neighborhood overlap-based heuristic methods and handle overlapped multi-hop neighborhoods. Our extensive experiments on Open Graph Benchmark datasets (OGB) demonstrate that Neo-GNNs consistently achieve state-of-the-art performance in link prediction. Our code is publicly available at.

Benchmark Results

Tested on 4 datasets 9 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

OGB

#ModelHits@50 ↑Paper
1
PROXI
0.7650 ±0.0027
2
OGB Leader
0.7129 ±0.0018
3
GIDN
0.7096 ±0.0055
9
GCN
actual #9 vs paper #85 76 places worse
0.6621 ±0.0033
best reported
actual #9 vs paper #85 76 places worse
paper 0.4701 ±0.0079
best at pub: OGB (NeurIPS 2020)
paper 0.4701 ±0.0079 Δ at pub 0.0013 Δ today 0.1920
20
SEAL
actual #20 vs paper #42 22 places worse
0.6474 ±0.0043
best reported
actual #20 vs paper #42 22 places worse
paper 0.5437 ±0.0002
28
SAGE
0.5944 ±0.0137
33
Neo-GNN (w/ GCN)
0.5752 ±0.0037
This paper
34
GAT
actual #34 vs paper #89 55 places worse
0.5643 ±0.0086
best reported
actual #34 vs paper #89 55 places worse
paper 0.4489 ±0.0123
paper 0.4489 ±0.0123 Δ today 0.1154
41
GraphSAGE
actual #41 vs paper #79 38 places worse
0.5463 ±0.0112
best reported
actual #41 vs paper #79 38 places worse
paper 0.4860 ±0.0046
best at pub: OGB (NeurIPS 2020)
paper 0.4860 ±0.0046 Δ at pub 0.0603 Δ today 0.0603
49
Adamic Adar
0.5300
This paper
50
Resource Allocation
0.5289
This paper
62
Common Neighbors
0.5006
This paper
68
Node2Vec
actual #68 vs paper #99 31 places worse
0.4929 ±0.0064
best reported
actual #68 vs paper #99 31 places worse
paper 0.4136 ±0.0069
paper 0.4136 ±0.0069 Δ at pub 0.0754 Δ today 0.0793
77
JK-Net
0.4884 ±0.0083
This paper
94
MLP
actual #94 vs paper #110 16 places worse
0.1998 ±0.0096
actual #94 vs paper #110 16 places worse
paper 0.1998 ±0.0096
prev best: OGB (NeurIPS 2020)

Improvement within reported uncertainty; not highlighted.

paper 0.1998 ±0.0096 Δ at pub 0.0071 Δ today 0.2440
103
Matrix Factorization
0.3874 ±0.0030
This paper
#ModelHits@20 ↑Paper
1
ELGNN
0.9777 ±0.0037
2
GIDN
0.9542
3
AGDN
0.9538 ±0.0094
20
SAGE
0.6719 ±0.0118
21
GAT
actual #21 vs paper #60 39 places worse
0.6683 ±0.0223
best reported
actual #21 vs paper #60 39 places worse
paper 0.2951 ±0.0640
paper 0.2951 ±0.0640 Δ today 0.3732
24
GCN
actual #24 vs paper #46 22 places worse
0.4460 ±0.0887
actual #24 vs paper #46 22 places worse
paper 0.4460 ±0.0887
paper 0.4460 ±0.0887 Δ at pub 0.0749 Δ today 0.2016
25
Node2Vec
actual #25 vs paper #72 47 places worse
0.6363 ±0.0205
best reported
actual #25 vs paper #72 47 places worse
paper 0.2195 ±0.0158
paper 0.2195 ±0.0158 Δ at pub 0.0135 Δ today 0.4168
26
Neo-GNN
0.6357 ±0.0352
This paper
32
JK-Net
0.5798 ±0.0688
This paper
38
GraphSAGE
actual #38 vs paper #44 6 places worse
0.5390 ±0.0474
best reported
actual #38 vs paper #44 6 places worse
paper 0.4801 ±0.0902
best at pub: OGB (NeurIPS 2020)

Difference within reported uncertainty; not counted as underperformance.

paper 0.4801 ±0.0902 Δ at pub 0.0589 Δ today 0.0589
43
SEAL
actual #43 vs paper #65 22 places worse
0.4974 ±0.0239
best reported
actual #43 vs paper #65 22 places worse
paper 0.2625 ±0.0600
paper 0.2625 ±0.0600 Δ at pub 0.0431 Δ today 0.2349
76
Adamic Adar
0.1861
This paper
78
Matrix Factorization
actual #78 vs paper #78
0.1792 ±0.0357
actual #78 vs paper #78
paper 0.1792 ±0.0357

Improvement within reported uncertainty; not highlighted.

paper 0.1792 ±0.0357 Δ at pub 0.0424
79
Common Neighbors
0.1773
This paper
92
Resource Allocation
0.0623
This paper
#ModelHits@100 ↑Paper
1
OGB Leader
0.6524 ±0.0150
2
PRB
0.6347 ±0.0175
3
LPFormer
0.6332 ±0.0063
14
SAGE
0.5013 ±0.0055
17
Resource Allocation
0.4933
This paper
18
Neo-GNN
0.4913 ±0.0060
This paper
19
SEAL
actual #19 vs paper #20 1 place worse
0.4880 ±0.0316
best reported
actual #19 vs paper #20 1 place worse
paper 0.4815 ±0.0417

Difference within reported uncertainty; not counted as underperformance.

paper 0.4815 ±0.0417 Δ at pub 0.0065 Δ today 0.0065
23
MLP
actual #23 vs paper #71 48 places worse
0.4600
best reported
actual #23 vs paper #71 48 places worse
paper 0.0047 ±0.0005
best at pub: OGB (NeurIPS 2020)
paper 0.0047 ±0.0005 Δ at pub 0.4553 Δ today 0.4553
40
Adamic Adar
0.3245
This paper
43
Matrix Factorization
actual #43 vs paper #48 5 places worse
0.3229 ±0.0094
best reported
actual #43 vs paper #48 5 places worse
paper 0.2783 ±0.0202
paper 0.2783 ±0.0202 Δ at pub 0.0446
45
GCN
actual #45 vs paper #62 17 places worse
0.3084 ±0.0178
best reported
actual #45 vs paper #62 17 places worse
paper 0.1698 ±0.0133
paper 0.1698 ±0.0133 Δ at pub 0.0172 Δ today 0.1386
52
Common Neighbors
0.2765
This paper
57
Node2Vec
actual #57 vs paper #62 5 places worse
0.2230 ±0.0080
best reported
actual #57 vs paper #62 5 places worse
paper 0.1724 ±0.0076
paper 0.1724 ±0.0076 Δ at pub 0.0506
63
GraphSAGE
actual #63 vs paper #64 1 place worse
0.1655 ±0.0240
best reported
actual #63 vs paper #64 1 place worse
paper 0.1393 ±0.0238
best at pub: OGB (NeurIPS 2020)
paper 0.1393 ±0.0238 Δ at pub 0.0262 Δ today 0.0262
67
JK-Net
0.1140 ±0.0204
This paper
#ModelMRR ↑Paper
1
CFG
0.8997 ±0.0015
2
SIEG
0.8987 ±0.0018
3
LPFormer
0.8981 ±0.0013
8
SEAL
actual #8 vs paper #14 6 places worse
0.8767 ±0.0032
best reported
actual #8 vs paper #14 6 places worse
paper 0.8632 ±0.0052
11
Neo-GNN
0.8726 ±0.0084
This paper
12
GCN
actual #12 vs paper #21 9 places worse
0.8479 ±0.0024
actual #12 vs paper #21 9 places worse
paper 0.8479 ±0.0024
prev best: OGB (NeurIPS 2020)

Improvement within reported uncertainty; not highlighted.

paper 0.8479 ±0.0024 Δ at pub 0.0005 Δ today 0.0226
30
SAGE
0.8292 ±0.0022
32
GraphSAGE
actual #32 vs paper #32
0.8264 ±0.0001
actual #32 vs paper #32
paper 0.8264 ±0.0001
prev best: OGB (NeurIPS 2020)

Improvement within reported uncertainty; not highlighted.

paper 0.8264 ±0.0001 Δ at pub 0.0004
48
Common Neighbors
0.7620
This paper
49
Resource Allocation
actual #49 vs paper #49
0.7620
actual #49 vs paper #49
paper 0.7620
paper 0.7620 Δ at pub 0.2422
50
Adamic Adar
actual #50 vs paper #50
0.7612
actual #50 vs paper #50
paper 0.7612
paper 0.7612 Δ at pub 0.2423
59
Node2Vec
actual #59 vs paper #60 1 place worse
0.6141 ±0.0011
best reported
actual #59 vs paper #60 1 place worse
paper 0.5347 ±0.0012
best at pub: OGB (NeurIPS 2020)
paper 0.5347 ±0.0012 Δ at pub 0.0794 Δ today 0.0794
61
Matrix Factorization
actual #61 vs paper #61
0.5308 ±0.0419
actual #61 vs paper #61
paper 0.5308 ±0.0419

Improvement within reported uncertainty; not highlighted.

paper 0.5308 ±0.0419 Δ at pub 0.0122
71
MLP
actual #71 vs paper #71
0.3917 ±0.0044
best reported
actual #71 vs paper #71
paper 0.2899 ±0.0016
SOTA performance (not significantly different from the best) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best at pub — reported value improved over the best known when published Improved at pub — later paper surpassed this result Best today — OK at publication; today's leaderboard best is elsewhere