UGSL: A Unified Framework for Benchmarking Graph Structure Learning

Bahare Fatemi, Sami Abu-El-Haija, Anton Tsitsulin, Seyed Mehran Kazemi, Dustin Zelle, Neslihan Bulut, Jonathan J. Halcrow, Bryan Perozzi

Abstract

Graph neural networks (GNNs) demonstrate outstanding performance in a broad range of applications. While the majority of GNN applications assume that a graph structure is given, some recent methods substantially expanded the applicability of GNNs by showing that they may be effective even when no graph structure is explicitly provided. The GNN parameters and a graph structure are jointly learned. Previous studies adopt different experimentation setups, making it difficult to compare their merits. In this paper, we propose a benchmarking strategy for graph structure learning using a unified framework. Our framework, called Unified Graph Structure Learning (UGSL), reformulates existing models into a single model. We implement a wide range of existing models in our framework and conduct extensive analyses of the effectiveness of different components in the framework. Our results provide a clear and concise understanding of the different methods in this area as well as their strengths and weaknesses. The benchmark code is available at.

Benchmark Results

Tested on 3 datasets 8 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

Cora

Global leaderboard →

#	Model	Accuracy ↑	Paper
1	GraphSAGE+CNA code ↗ GNN global	0.9418 ±0.0033	Graph Neural Networks Need Cluster-Normalize-Activate Modules code ↗
2	GCN+CNA code ↗ GNN global	0.9366 ±0.0048	Graph Neural Networks Need Cluster-Normalize-Activate Modules code ↗
3	TransformerConv code ↗ GNN global	0.9350 ±0.0058	Graph Neural Networks Need Cluster-Normalize-Activate Modules code ↗
21	GCN GNN baseline actual #21 vs paper #855 834 places worse	0.9004 best reported	actual #21 vs paper #855 834 places worse paper 0.6530 best at pub: Language is All a Graph Needs (2023) best today: Understanding Heterophily for Graph Neural Networks (ICML 2024) paper 0.6530 Δ at pub 0.2248 Δ today 0.2474
39	SAGE GNN global	0.8911 ±0.0012	Efficient Text-Attributed Graph Learning through Selective Annotation and Graph Alignment
165	GIN GNN baseline actual #165 vs paper #876 711 places worse	0.8660 ±0.0091 best reported	actual #165 vs paper #876 711 places worse paper 0.6020 best at pub: Weisfeiler and Leman Go Infinite: Spectral and Combinatorial Pre-Colorings (TMLR 2022) best today: Large Language Model Meets Graph Neural Network in Knowledge Distillation (AAAI 2024) paper 0.6020 Δ at pub 0.2160 Δ today 0.2640
724	MLP Trad baseline actual #724 vs paper #855 131 places worse	0.7768 best reported	actual #724 vs paper #855 131 places worse paper 0.6530 best at pub: From Node Interaction to Hop Interaction: New Effective and Scalable Graph Learning Paradigm (2022) best today: Understanding Heterophily for Graph Neural Networks (ICML 2024) paper 0.6530 Δ at pub 0.0945 Δ today 0.1238
805	Cora (best) baseline	0.7230	This paper
811	contrastive loss baseline	0.7140	This paper
819	denoising loss baseline	0.7021	This paper
831	sparse-connect, log-barrier Trad baseline	0.6860	This paper
836	Spectral Roles Trad baseline	0.6810	This paper
837	d-kNN Trad baseline	0.6800	This paper
838	Closeness Trad baseline	0.6800	This paper
841	FP GNN baseline	0.6770	This paper
843	sparse-connect Trad baseline	0.6750	This paper
846	activation-symmetrize baseline	0.6712	This paper
848	log-barrier Trad baseline	0.6690	This paper
849	symmetrize baseline	0.6680	This paper
850	smoothness Trad baseline	0.6620	This paper
851	activation baseline	0.6610	This paper
856	Raw features Trad baseline actual #856 vs paper #856	0.6530	actual #856 vs paper #856 paper 0.6530 prev best: HomoGCL: Rethinking Homophily in Graph Contrastive Learning (KDD 2023) paper 0.6530 Δ at pub 0.1760
858	KNN Trad baseline	0.6530	This paper
857	None GNN baseline	0.6530	This paper
859	one adjacency baseline	0.6530	This paper
860	per layer adjacency baseline	0.6522	This paper
861	psilonNN Trad baseline	0.6520	This paper
863	Random d-kNN Trad baseline	0.6500	This paper
895	ATT GNN baseline	0.5330	This paper
904	WL Roles Trad baseline	0.4960	This paper
905	Bernoulli relaxation Trad baseline	0.4540	This paper

CiteSeer

Global leaderboard →

#	Model	Accuracy ↑	Paper
1	CNA code ↗ GNN global	0.9575 ±0.0058	Graph Neural Networks Need Cluster-Normalize-Activate Modules code ↗
2	IS-GIB code ↗ Hyb global	0.9390 ±0.0187	Individual and Structural Graph Information Bottlenecks for Out-of-Distribution Generalization code ↗
3	EERM code ↗ GNN global	0.9112 ±0.0145	Individual and Structural Graph Information Bottlenecks for Out-of-Distribution Generalization code ↗
44	GCN GNN baseline actual #44 vs paper #708 664 places worse	0.8033 ±0.0088 best reported	actual #44 vs paper #708 664 places worse paper 0.6730 best at pub: Gophormer: Ego-Graph Transformer for Node Classification (2021) best today: NTFormer: A Composite Node Tokenized Graph Transformer for Node Classification (2024) paper 0.6730 Δ at pub 0.1213 Δ today 0.1303
61	GraphSAGE GNN global	0.7923 ±0.0053	Gophormer: Ego-Graph Transformer for Node Classification
350	MLP code ↗ Trad global	0.7289 ±0.0010	Gumbel-MPNN: Graph Rewiring with Gumbel-Softmax code ↗
389	GIN GNN baseline actual #389 vs paper #815 426 places worse	0.7250 ±0.0080 best reported	actual #389 vs paper #815 426 places worse paper 0.6201 best at pub: Towards Self-Explainable Graph Neural Network (2021) paper 0.6201 Δ at pub 0.1049
543	Citeseer baseline	0.7120	This paper
566	None GNN baseline actual #566 vs paper #708 142 places worse	0.7090 ±0.0070 best reported	actual #566 vs paper #708 142 places worse paper 0.6730 best at pub: AutoGL: A Library for Automated Graph Learning (ICLR 2021) paper 0.6730 Δ at pub 0.0360
628	FP GNN baseline	0.6990	This paper
673	contrastive loss baseline	0.6841	This paper
682	Spectral Roles Trad baseline	0.6810	This paper
698	sparse-connect, log-barrier Trad baseline	0.6770	This paper
707	sparse-connect Trad baseline	0.6740	This paper
709	Raw features Trad baseline actual #709 vs paper #709	0.6730	actual #709 vs paper #709 paper 0.6730 prev best: Generative Subgraph Contrast for Self-Supervised Graph Representation Learning (2022) paper 0.6730 Δ at pub 0.0950
710	KNN Trad baseline	0.6730	This paper
711	one adjacency baseline	0.6730	This paper
722	per layer adjacency baseline	0.6700	This paper
731	smoothness Trad baseline	0.6660	This paper
740	Closeness Trad baseline	0.6630	This paper
748	d-kNN Trad baseline	0.6620	This paper
749	Random d-kNN Trad baseline	0.6620	This paper
757	symmetrize baseline	0.6590	This paper
760	denoising loss baseline	0.6571	This paper
792	Bernoulli relaxation Trad baseline	0.6410	This paper
807	activation baseline	0.6300	This paper
813	activation-symmetrize baseline	0.6230	This paper
826	psilonNN Trad baseline	0.6060	This paper
854	log-barrier Trad baseline	0.5600	This paper
856	ATT GNN baseline	0.5440	This paper
893	WL Roles Trad baseline	0.2990	This paper

PubMed

Global leaderboard →

#	Model	Accuracy ↑	Paper
1	LGTL GT global	0.9518 ±0.0011	Adaptive Tokenization: On the Hop-Overpriority Problem in Tokenized Graph Learning Models
2	MixHop (LO) code ↗ GNN global	0.9513 ±0.0038	LOGIN: A Large Language Model Consulted Graph Neural Network Training Framework code ↗
3	GraphSAGE (LO) code ↗ GNN global	0.9511 ±0.0036	LOGIN: A Large Language Model Consulted Graph Neural Network Training Framework code ↗
18	GCN GNN baseline actual #18 vs paper #841 823 places worse	0.9296 ±0.0015 best reported	actual #18 vs paper #841 823 places worse paper 0.6960 best at pub: Tackling Over-Smoothing for General Graph Convolutional Networks (2020) best today: Adaptive Tokenization: On the Hop-Overpriority Problem in Tokenized Graph Learning Models (2025) paper 0.6960 Δ at pub 0.2010 Δ today 0.2336
39	contrastive loss baseline	0.9104	This paper
116	SAGE GNN global	0.8881 ±0.0015	Efficient Text-Attributed Graph Learning through Selective Annotation and Graph Alignment
157	GIN GNN baseline actual #157 vs paper #157	0.8810	actual #157 vs paper #157 paper 0.8810 prev best: Learning Conjoint Attentions for Graph Neural Nets (NeurIPS 2021) paper 0.8810 Δ at pub 0.0734
209	activation-symmetrize baseline	0.8693	This paper
210	MLP code ↗ Trad global	0.8690 ±0.0074	On the Equivalence of Graph Convolution and Mixup code ↗
215	symmetrize baseline	0.8680	This paper
233	sparse-connect Trad baseline	0.8593	This paper
258	Closeness Trad baseline	0.8500	This paper
279	smoothness Trad baseline	0.8400	This paper
309	sparse-connect, log-barrier Trad baseline	0.8284	This paper
337	activation baseline	0.8183	This paper
710	denoising loss baseline	0.7757	This paper
772	Pubmed (best) baseline	0.7600	This paper
823	FP GNN baseline	0.7220	This paper
828	Raw features Trad baseline actual #828 vs paper #841 13 places worse	0.7140 ±0.0020 best reported	actual #828 vs paper #841 13 places worse paper 0.6960 best at pub: HomoGCL: Rethinking Homophily in Graph Contrastive Learning (KDD 2023) paper 0.6960 Δ at pub 0.0180
830	Random d-kNN Trad baseline	0.7140	This paper
837	per layer adjacency baseline	0.7027	This paper
838	d-kNN Trad baseline	0.7010	This paper
842	KNN Trad baseline	0.6960	This paper
843	one adjacency baseline	0.6960	This paper
844	None GNN baseline	0.6960	This paper
853	Spectral Roles Trad baseline	0.6790	This paper
857	ATT GNN baseline	0.6650	This paper
860	psilonNN Trad baseline	0.6600	This paper
869	WL Roles Trad baseline	0.5850	This paper
878	Bernoulli relaxation Trad baseline	0.4130	This paper
883	log-barrier Trad baseline	0.4036	This paper

SOTA performance (not significantly different from the best) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best at pub — reported value improved over the best known when published