OpenGraph: Towards Open Graph Foundation Models

Lianghao Xia, Ben Kao, Chao Huang

Main figure for OpenGraph: Towards Open Graph Foundation Models

Abstract

Graph learning has become essential in various domains, including recommendation systems and social network analysis. Graph Neural Networks (GNNs) have emerged as promising techniques for encoding structural information and improving performance in tasks like link prediction and node classification. However, a key challenge remains: the difficulty of generalizing to unseen graph data with different properties. In this work, we propose a novel graph foundation model, called OpenGraph, to address this challenge. Our approach tackles several technical obstacles. Firstly, we enhance data augmentation using a large language model (LLM) to overcome data scarcity in real-world scenarios. Secondly, we introduce a unified graph tokenizer that enables the model to generalize effectively to diverse graph data, even when encountering unseen properties during training. Thirdly, our developed scalable graph transformer captures node-wise dependencies within the global topological context. Extensive experiments validate the effectiveness of our framework. By adapting OpenGraph\ to new graph characteristics and comprehending diverse graphs, our approach achieves remarkable zero-shot graph learning performance across various settings. We release the model implementation at https://github.com/HKUDS/OpenGraph.

Proposed method: OpenGraph

Benchmark Results

Tested on 3 datasets 12 baseline values reported too low Non-standard metric on 1 dataset

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

CiteSeer

Global leaderboard →

Default metric is Accuracy. The paper reports 10 values on Accuracy and 9 values on F1 .

#ModelAccuracy ↑ defaultF1 ↑Paper
1
CNA
0.9575 ± 0.0058
—
2
IS-GIB
0.9390 ± 0.0187
—
3
EERM
0.9112 ± 0.0145
—
35
GAT
0.8080 ±0.0026
best reported
0.2025
F1 0.2025
paper 0.2810
paper 0.2810 Δ at pub 0.5270
44
GCN
0.8033 ±0.0088
best reported
0.2096
F1 0.2096
paper 0.3200
61
GraphSAGE
0.7923 ± 0.0053
—
66
DGI
0.7890 ±0.0020
best reported
0.1791
F1 0.1791
paper 0.2840
paper 0.2840 Δ at pub 0.5050
350
MLP
0.7289 ±0.0010
best reported
0.1993
F1 0.1993
paper 0.2590
paper 0.2590 Δ at pub 0.4600 Δ today 0.4699
389
GIN
0.7250 ±0.0080
best reported
0.1705
F1 0.1705
paper 0.2820
paper 0.2820 Δ at pub 0.4430
395
GraphCL
0.7250 ±0.0070
best reported
0.1630
F1 0.1630
paper 0.2980
438
OpenGraph
0.7221
—
F1 —
This paper
861
GPrompt
0.5321 ±0.1094
best reported
0.1519
F1 0.1519
paper 0.2800
best today: ProG: A Graph Prompt Learning Benchmark (NeurIPS 2024)
paper 0.2800 Δ today 0.2521
869
GPF
0.4902 ±0.0453
best reported
0.1984
F1 0.1984
paper 0.3110
902
MF
0.1740
0.1727
F1 0.1727
This paper

PubMed

Global leaderboard →

Default metric is Accuracy. The paper reports 10 values on Accuracy and 10 values on F1 .

#ModelAccuracy ↑ defaultF1 ↑Paper
1
LGTL
0.9518 ± 0.0011
—
2
MixHop (LO)
0.9513 ± 0.0038
—
3
GraphSAGE (LO)
0.9511 ± 0.0036
—
18
GCN
0.9296 ±0.0015
best reported
0.3296
F1 0.3296
paper 0.4270
20
GAT
0.9233 ±0.0018
best reported
0.3657
F1 0.3657
paper 0.4720
116
SAGE
0.8881 ± 0.0015
—
157
GIN
0.8810
best reported
0.3064
F1 0.3064
paper 0.4410
paper 0.4410 Δ at pub 0.4400
204
DGI
0.8708 ±0.0017
best reported
0.3163
210
MLP
0.8690 ±0.0074
best reported
0.3114
F1 0.3114
paper 0.4430
paper 0.4430 Δ at pub 0.4260
331
GraphCL
0.8196 ±0.0073
best reported
0.4130
F1 0.4130
paper 0.4070
paper 0.4070 Δ at pub 0.4126
847
OpenGraph
0.6869
0.6537
F1 0.6537
This paper
870
GPF
0.5716 ±0.1099
best reported
0.2670
F1 0.2670
paper 0.4220

Difference within reported uncertainty; not counted as underperformance.

paper 0.4220 Δ today 0.1496
874
GPrompt
0.4710
0.3705
F1 0.3705
This paper
886
MF
0.3470
0.3346
F1 0.3346
This paper

OGB

ogbl-ddi

Global leaderboard →
Non-default metric. This paper reports Recall@20; the dataset default is Hits@20. Values below use Recall@20 only.

Sorted by Recall@20. Highlighted rows are from other papers.

#ModelRecall@20 ↑Paper
1
OpenGraph
0.0921
This paper
2
GPF
0.0839
This paper
3
DGI
0.0821
This paper
4
GPrompt
0.0769
This paper
5
GraphCL
0.0740
This paper
6
GIN
0.0735
This paper
7
GAT
0.0711
This paper
8
GCN
0.0705
This paper
9
MLP
0.0621
This paper
10
MF
0.0536
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best today — OK at publication; today's leaderboard best is elsewhere