On the Safety of Graph Representation Learning

Xiaoguang Guo, Zehong Wang, Ziming Li, Shawn Spitzel, S. Kwon, Tianyi Ma, Yanfang Ye, Chuxu Zhang

Main figure for On the Safety of Graph Representation Learning

Abstract

Graph representation learning (GRL) has evolved from topology-only graph embeddings to task-specific supervised GNNs, and more recently to reusable representations and graph foundation models (GFMs). However, existing evaluations mainly measure clean transfer, adaptation, and task coverage. It remains unclear whether GRL methods stay reliable when deployment stresses affect graph signals, graph contexts, label support, structural groups, or predictive evidence. We introduce GRL-Safety, a multi-axis safety evaluation benchmark for GRL. GRL-Safety evaluates twelve representative methods, spanning topology-only embedding methods, supervised GNNs, self-supervised graph models, and GFMs, on twenty-five graph datasets under standardized evaluation conditions while preserving method-native adaptation. The evaluation covers five safety axes: corruption robustness, OOD generalization, class imbalance, fairness, and interpretation, with per-axis and subcondition reporting rather than a single aggregate score. Our analysis yields three cross-axis insights that can inspire future research. First, safety behavior is shaped by the interaction between representation design and the stressed graph factor, rather than by method family alone. Second, foundation-era methods show axis-specific strengths rather than broad safety dominance. Third, several deployment regimes remain difficult even for the best evaluated method, revealing capability gaps that require new robustness, adaptation, or training objectives beyond model selection. The benchmark, evaluation protocols, and code are available at:.

Benchmark Results

Tested on 2 datasets 7 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

Heterophily Benchmark

#ModelROC-AUC ↑Paper
1
Polynormer-r
0.8591 ±0.0074
2
DC-GNN
0.8588 ±0.0081
3
TANGO-GPS
0.8566 ±0.0101
28
GAT
actual #28 vs paper #150 122 places worse
0.8443 ±0.0100
best reported
actual #28 vs paper #150 122 places worse
paper 0.7001 ±0.0227
paper 0.7001 ±0.0227 Δ at pub 0.1442
50
GCN
actual #50 vs paper #136 86 places worse
0.8364 ±0.0067
best reported
actual #50 vs paper #136 86 places worse
paper 0.7478 ±0.0031
paper 0.7478 ±0.0031 Δ at pub 0.0886 Δ today 0.0886
76
SAGE
actual #76 vs paper #135 59 places worse
0.8243 ±0.0044
best reported
actual #76 vs paper #135 59 places worse
paper 0.7559 ±0.0045
paper 0.7559 ±0.0045 Δ at pub 0.0684 Δ today 0.0684
77
GraphSAGE
0.8243 ±0.0044
93
BGRL
0.8041 ±0.0036
This paper
130
GIT
0.7666 ±0.0044
This paper
139
MLP
0.7362 ±0.0020
145
GFT
0.7242 ±0.0061
This paper
152
UniGraph2
0.6934 ±0.0520
This paper
153
GraphMAE
0.6890 ±0.0346
This paper

Knowledge Graphs

#ModelMRR ↑Paper
1
FedSage
0.8426 ±0.0026
2
FED-PUB
0.8348 ±0.0025
3
FedAvg
0.8252 ±0.0023
4
GAT
actual #4 vs paper #106 102 places worse
0.8244 ±0.0037
best reported
actual #4 vs paper #106 102 places worse
paper 0.1670 ±0.0110
7
GCN
actual #7 vs paper #106 99 places worse
0.8171 ±0.0030
best reported
actual #7 vs paper #106 99 places worse
paper 0.1920 ±0.0140
8
GraphSAGE
actual #8 vs paper #109 101 places worse
0.8152 ±0.0024
best reported
actual #8 vs paper #109 101 places worse
paper 0.1340 ±0.0270
9
GIN
0.8124 ±0.0055
10
OFA
actual #10 vs paper #110 100 places worse
0.8000
best reported
actual #10 vs paper #110 100 places worse
paper 0.0750 ±0.0110
paper 0.0750 ±0.0110 Δ at pub 0.7250
101
BGRL
0.2310 ±0.0090
This paper
102
GraphMAE
0.2240 ±0.0140
This paper
107
GFT
0.1570 ±0.0050
This paper
108
LLaGA
0.1520 ±0.0190
This paper
109
UniGraph2
0.1480 ±0.0070
This paper
110
GIT
0.0940 ±0.0130
This paper
SOTA performance (not significantly different from the best) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green)