ICLR 2024

Fully-inductive Node Classification on Arbitrary Graphs

Jianan Zhao, Hesham Mostafa, Mikhail Galkin, Michael M. Bronstein, Zhaocheng Zhu, Jian Tang

Main figure for Fully-inductive Node Classification on Arbitrary Graphs

Abstract

One fundamental challenge in graph machine learning is generalizing to new graphs. Many existing methods following the inductive setup can generalize to test graphs with new structures, but assuming the feature and label spaces remain the same as the training ones. This paper introduces a fully-inductive setup, where models should perform inference on arbitrary test graphs with new structures, feature and label spaces. We propose GraphAny as the first attempt at this challenging setup. GraphAny models inference on a new graph as an analytical solution to a LinearGNN, which can be naturally applied to graphs with any feature and label spaces. To further build a stronger model with learning capacity, we fuse multiple LinearGNN predictions with learned inductive attention scores. Specifically, the attention module is carefully parameterized as a function of the entropy-normalized distance features between pairs of LinearGNN predictions to ensure generalization to new graphs. Empirically, GraphAny trained on a single Wisconsin dataset with only 120 labeled nodes can generalize to 30 new graphs with an average accuracy of 67.26\

Benchmark Results

Tested on 6 datasets 11 baseline values reported too low Non-standard metric on 2 datasets

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

#ModelAccuracy ↑Paper
1
GraphSAGE+CNA
0.9418 ±0.0033
2
GCN+CNA
0.9366 ±0.0048
3
TransformerConv
0.9350 ±0.0058
21
GCN
actual #21 vs paper #578 557 places worse
0.9004
best reported
actual #21 vs paper #578 557 places worse
paper 0.8140 ±0.0070
paper 0.8140 ±0.0070 Δ at pub 0.0864
39
SAGE
0.8911 ±0.0012
44
GAT
actual #44 vs paper #558 514 places worse
0.8897 ±0.0014
best reported
actual #44 vs paper #558 514 places worse
paper 0.8170 ±0.0143
paper 0.8170 ±0.0143 Δ at pub 0.0720 Δ today 0.0727
712
LinearSGC2
0.7820
This paper
724
MLP
actual #724 vs paper #904 180 places worse
0.7768
best reported
actual #724 vs paper #904 180 places worse
paper 0.4842 ±0.0063
paper 0.4842 ±0.0063 Δ at pub 0.2926
784
LinearSGC1
0.7430
This paper
802
Linear
actual #802 vs paper #896 94 places worse
0.7237 ±0.0033
best reported
actual #802 vs paper #896 94 places worse
paper 0.5280
876
LabelProp
0.6030
This paper
919
LinearHGC2
0.2380
This paper
921
LinearHGC1
0.2250
This paper

Reddit

Global leaderboard →
Non-default metric. This paper reports Accuracy; the dataset default is F1. Values below use Accuracy only.

Sorted by Accuracy. Highlighted rows are from other papers.

#ModelAccuracy ↑Paper
1
GAP-EDP
0.9870 ±0.0003
2
NE-ASGCN
0.9758 ±0.0027
3
HSGT
0.9730 ±0.0024
21
GraphSAGE
0.9653 ±0.0011
22
GAT
actual #22 vs paper #97 75 places worse
0.9650 ±0.0011
best reported
actual #22 vs paper #97 75 places worse
paper 0.9276 ±0.0046
paper 0.9276 ±0.0046 Δ at pub 0.0374 Δ today 0.0374
26
GCN
actual #26 vs paper #73 47 places worse
0.9648
best reported
actual #26 vs paper #73 47 places worse
paper 0.9489 ±0.0002
paper 0.9489 ±0.0002 Δ at pub 0.0159
114
GraphAny (Wisconsin)
0.9100 ±0.0024
This paper
147
MLP
actual #147 vs paper #150 3 places worse
0.8240 ±0.0010
best reported
actual #147 vs paper #150 3 places worse
paper 0.7216 ±0.0015
paper 0.7216 ±0.0015 Δ at pub 0.1024
#ModelAccuracy ↑Paper
1
GCN
actual #1 vs paper #128 127 places worse
0.9302 ±0.0011
best reported
actual #1 vs paper #128 127 places worse
paper 0.7912 ±0.0045
paper 0.7912 ±0.0045 Δ at pub 0.1390
2
RSE + RGCN (70b)
0.8852 ±0.0019
3
LEMP+T
0.8825
9
SAGE
0.8771
31
GAT
actual #31 vs paper #118 87 places worse
0.8533
best reported
actual #31 vs paper #118 87 places worse
paper 0.7927 ±0.0020
232
GraphAny (Products)
0.7501 ±0.0054
This paper
237
MLP
actual #237 vs paper #248 11 places worse
0.7272 ±0.0043
actual #237 vs paper #248 11 places worse
paper 0.7272 ±0.0043
paper 0.7272 ±0.0043 Δ at pub 0.0074 Δ today 0.0197

OGB

#ModelAccuracy ↑Paper
1
Jacobi
0.8968
2
GPR
0.8953
3
LEMP4HG
0.8939
4
SAGE
0.8933
6
GCN
actual #6 vs paper #248 242 places worse
0.8868
best reported
actual #6 vs paper #248 242 places worse
paper 0.7579 ±0.0012
9
GAT
actual #9 vs paper #164 155 places worse
0.8829
best reported
actual #9 vs paper #164 155 places worse
paper 0.7945 ±0.0059
351
MLP
actual #351 vs paper #370 19 places worse
0.6605 ±0.0020
best reported
actual #351 vs paper #370 19 places worse
paper 0.6106 ±0.0008
paper 0.6106 ±0.0008 Δ at pub 0.2312 Δ today 0.0499
369
GraphAny
0.6119 ±0.0023
This paper
#ModelAccuracy ↑Paper
1
GIANT-XRT
0.7612 ±0.0016
2
GEGCN
0.7576 ±0.0064
3
AGDN
0.7522 ±0.0007
15
GAT
actual #15 vs paper #15
0.7365 ±0.0011
actual #15 vs paper #15
paper 0.7365 ±0.0011
paper 0.7365 ±0.0011 Δ at pub 0.0183
23
GCN
actual #23 vs paper #52 29 places worse
0.7300 ±0.0017
best reported
actual #23 vs paper #52 29 places worse
paper 0.7174 ±0.0029
25
GraphSAGE
0.7277 ±0.0016
107
MLP
actual #107 vs paper #108 1 place worse
0.5765 ±0.0012
best reported
actual #107 vs paper #108 1 place worse
paper 0.5550 ±0.0023

Heterophily Benchmark

Questions

Global leaderboard →
Non-default metric. This paper reports Accuracy; the dataset default is ROC-AUC. Values below use Accuracy only.

Sorted by Accuracy. Highlighted rows are from other papers.

#ModelAccuracy ↑Paper
1
MLP
0.9733
This paper
2
AGS-GNN
0.9727
3
MeanGNN
0.9716 ±0.0006
4
GCN
actual #4 vs paper #4
0.9715
actual #4 vs paper #4
paper 0.9715
paper 0.9715 Δ at pub 0.2106
5
GAT
actual #5 vs paper #5
0.9711
actual #5 vs paper #5
paper 0.9711
paper 0.9711 Δ at pub 0.1968
7
GraphAny
0.9710
This paper
33
SAGE
0.7644
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best at pub — reported value improved over the best known when published Improved at pub — later paper surpassed this result Best today — OK at publication; today's leaderboard best is elsewhere