NeurIPS 2023

Long-range Meta-path Search on Large-scale Heterogeneous Graphs

Chao Li, Zijie Guo, Qiuting He, Hao Xu, Kun He

Main figure for Long-range Meta-path Search on Large-scale Heterogeneous Graphs

Abstract

Utilizing long-range dependency, a concept extensively studied in homogeneous graphs, remains underexplored in heterogeneous graphs, especially on large ones, posing two significant challenges: Reducing computational costs while maximizing effective information utilization in the presence of heterogeneity, and overcoming the over-smoothing issue in graph neural networks. To address this gap, we investigate the importance of different meta-paths and introduce an automatic framework for utilizing long-range dependency on heterogeneous graphs, denoted as Long-range Meta-path Search through Progressive Sampling (LMSPS). Specifically, we develop a search space with all meta-paths related to the target node type. By employing a progressive sampling algorithm, LMSPS dynamically shrinks the search space with hop-independent time complexity. Through a sampling evaluation strategy, LMSPS conducts a specialized and effective meta-path selection, leading to retraining with only effective meta-paths, thus mitigating costs and over-smoothing. Extensive experiments across diverse heterogeneous datasets validate LMSPS's capability in discovering effective long-range meta-paths, surpassing state-of-the-art methods. Our code is available at.

Benchmark Results

Tested on 5 datasets 4 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

OGB

#ModelAccuracy ↑Paper
1
LMSPS+LP+MS
0.5784 ±0.0022
This paper
2
SeHGNN+emb+ms
0.5719 ±0.0012
4
SeHGNN+LP+MS
0.5671 ±0.0014
This paper
3
SeHGNN+ms
0.5671 ±0.0014
9
LMSPS+LP
0.5510 ±0.0011
This paper
10
LMSPS
0.5483 ±0.0020
This paper
19
SeHGNN
actual #19 vs paper #36 17 places worse
0.5399 ±0.0018
best reported
actual #19 vs paper #36 17 places worse
paper 0.5145 ±0.0029
paper 0.5145 ±0.0029 Δ at pub 0.0254
20
SeHGNN+LP
0.5399 ±0.0018
This paper
28
NARS
actual #28 vs paper #39 11 places worse
0.5240 ±0.0016
best reported
actual #28 vs paper #39 11 places worse
paper 0.5066 ±0.0022
paper 0.5066 ±0.0022 Δ at pub 0.0174 Δ today 0.0174
35
GAMLP+LP
0.5163 ±0.0022
This paper
38
SAGN+LP
0.5117 ±0.0032
This paper
42
HGT
actual #42 vs paper #56 14 places worse
0.5023 ±0.0048
best reported
actual #42 vs paper #56 14 places worse
paper 0.4678 ±0.0042
48
RGCN
actual #48 vs paper #53 5 places worse
0.4811 ±0.0048
best reported
actual #48 vs paper #53 5 places worse
paper 0.4737 ±0.0048

Difference within reported uncertainty; not counted as underperformance.

paper 0.4737 ±0.0048 Δ at pub 0.0059 Δ today 0.0074
57
GraphSAGE
0.4646 ±0.0060
67
Random
actual #67 vs paper #71 4 places worse
0.3707 ±0.0020
best reported
actual #67 vs paper #71 4 places worse
paper 0.3514 ±0.0378

Difference within reported uncertainty; not counted as underperformance.

paper 0.3514 ±0.0378 Δ today 0.0193
76
MLP
0.3243

Heterogeneous Graph Benchmarks

#ModelMicro-F1 ↑Paper
1
LMSPS
0.9566 ±0.0020
This paper
2
PSHGCN
0.9561 ±0.0012
3
RpHGNN
0.9555 ±0.0029
5
SeHGNN
actual #5 vs paper #6 1 place worse
0.9542 ±0.0017
best reported
actual #5 vs paper #6 1 place worse
paper 0.9524 ±0.0013

Difference within reported uncertainty; not counted as underperformance.

paper 0.9524 ±0.0013 Δ at pub 0.0018 Δ today 0.0018
6
SlotGAT
0.9531 ±0.0019
This paper
8
PMMM
0.9514 ±0.0022
This paper
19
space4HGNN
0.9463 ±0.0040
This paper
20
NARS
actual #20 vs paper #20
0.9461 ±0.0042
actual #20 vs paper #20
paper 0.9461 ±0.0042
paper 0.9461 ±0.0042 Δ at pub 0.1421
21
GCN
0.9460 ±0.0031
27
GraphMSE
0.9444 ±0.0013
This paper
30
DiffMG
0.9420 ±0.0036
This paper
34
Random
0.9384 ±0.0053
This paper
176
MLP
0.6447 ±0.0136
#ModelMicro-F1 ↑Paper
1
LMSPS
0.9469 ±0.0036
This paper
2
PSHGCN
0.9427 ±0.0023
3
SlotGAT
0.9406 ±0.0022
This paper
5
SeHGNN
actual #5 vs paper #5
0.9398 ±0.0036
best reported
actual #5 vs paper #5
paper 0.9387 ±0.0050

Difference within reported uncertainty; not counted as underperformance.

paper 0.9387 ±0.0050 Δ at pub 0.0011 Δ today 0.0011
7
HINormer
0.9383 ±0.0045
This paper
8
PMMM
0.9371 ±0.0017
This paper
14
NARS
0.9331 ±0.0033
This paper
23
GraphMSE
0.9254 ±0.0014
This paper
25
space4HGNN
0.9238 ±0.0010
This paper
29
GCN
0.9229
50
Random
0.9082 ±0.0093
This paper
78
DiffMG
0.8807 ±0.0304
This paper
109
MLP
0.7102 ±0.0221
#ModelMicro-F1 ↑Paper
1
ELLA
0.7930 ±0.0010
2
MHGCN
0.7820 ±0.0138
3
PSHGCN
0.7446 ±0.0032
5
SeHGNN
actual #5 vs paper #13 8 places worse
0.7342 ±0.0047
best reported
actual #5 vs paper #13 8 places worse
paper 0.6821 ±0.0032
paper 0.6821 ±0.0032 Δ at pub 0.0521
10
LMSPS
0.6870 ±0.0026
This paper
11
SlotGAT
0.6864 ±0.0033
This paper
14
GCN
0.6813 ±0.0083
16
HINormer
0.6783 ±0.0034
This paper
19
PMMM
0.6758 ±0.0022
This paper
28
NARS
actual #28 vs paper #28
0.6618 ±0.0070
actual #28 vs paper #28
paper 0.6618 ±0.0070
paper 0.6618 ±0.0070 Δ at pub 0.0878
39
space4HGNN
0.6396 ±0.0043
This paper
46
GraphMSE
0.6237 ±0.0103
This paper
56
DiffMG
0.5975 ±0.0123
This paper
74
MLP
0.5514
77
Random
0.5383 ±0.0066
This paper
#ModelMacro-F1 ↑Paper
1
ConCH
0.6475
2
HetGNN
0.6292
3
GC-HGNN
0.6247 ±0.0042
7
GCN
0.6090
21
LMSPS
0.5326 ±0.0047
This paper
22
HINormer
0.5218 ±0.0039
This paper
23
SeHGNN
actual #23 vs paper #25 2 places worse
0.5187 ±0.0086
best reported
actual #23 vs paper #25 2 places worse
paper 0.5071 ±0.0044

Difference within reported uncertainty; not counted as underperformance.

paper 0.5071 ±0.0044 Δ at pub 0.0116
26
NARS
0.4998 ±0.0177
This paper
27
SlotGAT
0.4968 ±0.0197
This paper
35
space4HGNN
0.4137 ±0.0449
This paper
43
Random
0.2122 ±0.0258
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best at pub — reported value improved over the best known when published