On the Safety of Graph Representation Learning
Abstract
Graph representation learning (GRL) has evolved from topology-only graph embeddings to task-specific supervised GNNs, and more recently to reusable representations and graph foundation models (GFMs). However, existing evaluations mainly measure clean transfer, adaptation, and task coverage. It remains unclear whether GRL methods stay reliable when deployment stresses affect graph signals, graph contexts, label support, structural groups, or predictive evidence. We introduce GRL-Safety, a multi-axis safety evaluation benchmark for GRL. GRL-Safety evaluates twelve representative methods, spanning topology-only embedding methods, supervised GNNs, self-supervised graph models, and GFMs, on twenty-five graph datasets under standardized evaluation conditions while preserving method-native adaptation. The evaluation covers five safety axes: corruption robustness, OOD generalization, class imbalance, fairness, and interpretation, with per-axis and subcondition reporting rather than a single aggregate score. Our analysis yields three cross-axis insights that can inspire future research. First, safety behavior is shaped by the interaction between representation design and the stressed graph factor, rather than by method family alone. Second, foundation-era methods show axis-specific strengths rather than broad safety dominance. Third, several deployment regimes remain difficult even for the best evaluated method, revealing capability gaps that require new robustness, adaptation, or training objectives beyond model selection. The benchmark, evaluation protocols, and code are available at:.
Benchmark Results
Tested on 2 datasets 7 baseline values reported too low
Tables show presented results in global context (best 3 papers + GCN + MLP).
Heterophily Benchmark
| # | Model | ROC-AUC ↑ | Paper |
|---|---|---|---|
| 1 | Polynormer-r | 0.8591
±0.0074 | |
| 2 | DC-GNN | 0.8588
±0.0081 | |
| 3 | TANGO-GPS | 0.8566
±0.0101 | |
| 28 | GAT | 0.8443
±0.0100 best reported | best at pub: TopoBench: A Framework for Benchmarking Topological Deep Learning (2024) |
| 50 | GCN | 0.8364
±0.0067 best reported | best at pub: Graph Neural Networks for Graphs with Heterophily: A Survey (2022) |
| 76 | SAGE | 0.8243
±0.0044 best reported | best at pub: Graph Neural Networks for Graphs with Heterophily: A Survey (2022) |
| 77 | GraphSAGE | 0.8243
±0.0044 | |
| 93 | BGRL | 0.8041
±0.0036 | This paper |
| 130 | GIT | 0.7666
±0.0044 | This paper |
| 139 | MLP | 0.7362
±0.0020 | |
| 145 | GFT | 0.7242
±0.0061 | This paper |
| 152 | UniGraph2 | 0.6934
±0.0520 | This paper |
| 153 | GraphMAE | 0.6890
±0.0346 | This paper |
Knowledge Graphs
| # | Model | MRR ↑ | Paper |
|---|---|---|---|
| 1 | FedSage | 0.8426
±0.0026 | |
| 2 | FED-PUB | 0.8348
±0.0025 | |
| 3 | FedAvg | 0.8252
±0.0023 | |
| 4 | GAT | 0.8244
±0.0037 best reported | |
| 7 | GCN | 0.8171
±0.0030 best reported | |
| 8 | GraphSAGE | 0.8152
±0.0024 best reported | |
| 9 | GIN | 0.8124
±0.0055 | |
| 10 | OFA | 0.8000 best reported | |
| 101 | BGRL | 0.2310
±0.0090 | This paper |
| 102 | GraphMAE | 0.2240
±0.0140 | This paper |
| 107 | GFT | 0.1570
±0.0050 | This paper |
| 108 | LLaGA | 0.1520
±0.0190 | This paper |
| 109 | UniGraph2 | 0.1480
±0.0070 | This paper |
| 110 | GIT | 0.0940
±0.0130 | This paper |
