GNNBenchmark
Abstract
In the last few years, graph neural networks (GNNs) have become the standard toolkit for analyzing and learning from data on graphs. This emerging field has witnessed an extensive growth of promising techniques that have been applied with success to computer science, mathematics, biology, physics and chemistry. But for any successful field to become mainstream and reliable, benchmarks must be developed to quantify progress. This led us in March 2020 to release a benchmark framework that i) comprises of a diverse collection of mathematical and real-world graphs, ii) enables fair model comparison with the same parameter budget to identify key architectures, iii) has an open-source, easy-to-use and reproducible code infrastructure, and iv) is flexible for researchers to experiment with new theoretical ideas. As of December 2022, the GitHub repository. has reached 2,000 stars and 380 forks, which demonstrates the utility of the proposed open-source framework through the wide usage by the GNN community. In this paper, we present an updated version of our benchmark with a concise presentation of the aforementioned framework characteristics, an additional medium-sized molecular dataset AQSOL, similar to the popular ZINC, but with a real-world measured chemical target, and discuss how this framework can be leveraged to explore new GNN designs and insights. As a proof of value of our benchmark, we study the case of graph positional encoding (PE) in GNNs, which was introduced with this benchmark and has since spurred interest of exploring more powerful PE for Transformers and GNNs in a robust experimental setting.
Benchmark Results
Tested on 11 datasets 18 baseline values reported too low
Tables show presented results in global context (best 3 papers + GCN + MLP).
Classic
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | GCN | 0.9302
±0.0011 best reported | best at pub: Augmentation-Free Graph Contrastive Learning with Performance Guarantee (2022) |
| 2 | RSE + RGCN (70b) | 0.8852
±0.0019 | |
| 3 | LEMP+T | 0.8825 | |
| 9 | SAGE | 0.8771 | |
| 31 | GAT | 0.8533 best reported | best at pub: Learning Graph Augmentations to Learn Graph Representations (2022) |
| 54 | GIN | 0.8303
±0.0021 best reported | |
| 69 | GraphSAGE | 0.8146
±0.0010 best reported | |
| 193 | MoNet-PE | 0.7748
±0.0071 | This paper |
| 194 | MoNet | 0.7743
±0.0067 | This paper |
| 206 | vanilla GCN | 0.7710
±0.0083 | This paper |
| 237 | MLP | 0.7469
±0.0085 best reported | best at pub: Augmentation-Free Graph Contrastive Learning with Performance Guarantee (2022) best today: View Space: Learning Representation across Arbitrary Graphs (ICML 2025) |
| # | Model | MAE ↓ | Paper |
|---|---|---|---|
| 1 | Ours(100K) | 0.9200
±0.0090 | |
| 3 | GatedGCN-E-PE | 0.9960
±0.0080 | This paper |
| 2 | GatedGCN-PE/E(500K) | 0.9960
±0.0080 | |
| 4 | 3WLGNN-E | 1.0420
±0.0640 | This paper |
| 8 | 3WLGNN | 1.1540
±0.0500 | This paper |
| 10 | GAT | 1.2400
±0.0110 best reported | best today: Graph Alignment for Benchmarking Graph Neural Networks and Learning Positional Encodings (ICML 2025) |
| 13 | GCN | 1.2800
±0.0110 best reported | best today: Graph Alignment for Benchmarking Graph Neural Networks and Learning Positional Encodings (ICML 2025) |
| 14 | GatedGCN-E | 1.2950
±0.0160 | This paper |
| 17 | GatedGCN | 1.3520
±0.0340 | This paper |
| 18 | MoNet | 1.3950
±0.0270 | This paper |
| 19 | GraphSAGE | 1.4310
±0.0100 | This paper |
| 20 | vanilla GCN | 1.4830
±0.0140 | This paper |
| 22 | MLP | 1.7440
±0.0160 | This paper |
| 23 | GIN | 1.8940
±0.0240 | This paper |
| 24 | RingGNN-E | 3.7690
±1.0120 | This paper |
| 25 | RingGNN | 20.2640
±7.5490 | This paper |
TU Dortmund
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | MSH-GNN | 0.9410
±0.0330 | |
| 2 | GRDL | 0.8260
±0.0120 | |
| 3 | GMN | 0.8225 | |
| 32 | GIN | 0.7880
±0.0410 best reported | best at pub: Breaking the Limits of Message Passing Graph Neural Networks (ICML 2021) best today: Permutation-Invariant graph partitioning: How graph neural networks capture structural interactions? (2023) Difference within reported uncertainty; not counted as underperformance. |
| 49 | GAT | 0.7628
±0.0241 | prev best: Breaking the Limits of Message Passing Graph Neural Networks (ICML 2021) Improvement within reported uncertainty; not highlighted. |
| 67 | SAGE | 0.7726
±0.0228 | |
| 95 | GatedGCN | 0.7672
±0.0311 | This paper |
| 111 | MoNet | 0.7645
±0.0289 | This paper |
| 144 | vanilla GCN | 0.7610
±0.0241 | This paper |
| 149 | MLP | 0.7582
±0.0291 | prev best: Breaking the Limits of Message Passing Graph Neural Networks (ICML 2021) Improvement within reported uncertainty; not highlighted. |
| 155 | GraphSAGE | 0.7556
±0.0191 | prev best: Graph Convolutional Networks with EigenPooling (KDD 2019) |
| 472 | RingGNN | 0.6756
±0.0755 | This paper |
| 485 | 3WLGNN | 0.6468
±0.0588 | This paper |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | GMN | 0.7866 | |
| 2 | MemGNN | 0.7550 | |
| 3 | TFGW SP | 0.7510
±0.0500 | |
| 17 | GatedGCN | 0.7000
±0.0494 | This paper |
| 20 | GAT | 0.6850
±0.0524 | prev best: Breaking the Limits of Message Passing Graph Neural Networks (ICML 2021) |
| 26 | GraphSAGE | 0.6817
±0.0545 | prev best: Graph Neural Networks With Convolutional ARMA Filters (2019) Improvement within reported uncertainty; not highlighted. |
| 32 | GIN | 0.6767
±0.0583 | prev best: Template based Graph Neural Network with Optimal Transport Distances (NeurIPS 2022) |
| 36 | GCN | 0.6650
±0.0691 | |
| 42 | vanilla GCN | 0.6583
±0.0461 | This paper |
| 67 | MoNet | 0.6300
±0.0809 | This paper |
| 85 | 3WLGNN | 0.6100
±0.0680 | This paper |
| 124 | MLP | 0.5583
±0.0352 | prev best: Breaking the Limits of Message Passing Graph Neural Networks (ICML 2021) |
| 172 | RingGNN | 0.4533
±0.0452 | This paper |
OGB
| # | Model | Hits@50 ↑ | Paper |
|---|---|---|---|
| 1 | PROXI | 0.7650
±0.0027 | |
| 2 | OGB Leader | 0.7129
±0.0018 | |
| 3 | GIDN | 0.7096
±0.0055 | |
| 9 | GCN | 0.4896
±0.0114 | prev best: OGB (NeurIPS 2020) best today: On the Impact of Feature Heterophily on Link Prediction with Graph Neural Networks (NeurIPS 2024) Improvement within reported uncertainty; not highlighted. |
| 28 | SAGE | 0.5944
±0.0137 | |
| 34 | GAT | 0.5150
±0.0096 | prev best: Neo-GNNs: Neighborhood Overlap-aware Graph Neural Networks for Link Prediction (NeurIPS 2022) best today: Asymmetric Learning for Graph Neural Network based Link Prediction (2023) |
| 41 | GraphSAGE | 0.5463
±0.0112 best reported | best at pub: OGB (NeurIPS 2020) |
| 51 | GatedGCN-PE | 0.5285
±0.0135 | This paper |
| 53 | GatedGCN | 0.5263
±0.0117 | This paper |
| 59 | vanilla GCN | 0.5042
±0.0113 | This paper |
| 71 | GatedGCN-E | 0.4921
±0.0156 | This paper |
| 94 | MLP | 0.2035
±0.0217 | prev best: Neo-GNNs: Neighborhood Overlap-aware Graph Neural Networks for Link Prediction (NeurIPS 2022) best today: Weak Models Can be Good Teachers: A Case Study on Link Prediction with MLPs (2025) Improvement within reported uncertainty; not highlighted. |
| 95 | Matrix Fact. | 0.4421
±0.0045 | This paper |
| 99 | GIN | 0.4173
±0.0228 | This paper |
| 105 | MoNet | 0.3614
±0.0219 | This paper |
GNNBenchmark
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | Schrödinger | 0.9913
±0.0004 | |
| 2 | CNN | 0.9907
±0.0007 | |
| 3 | MPNN | 0.9895
±0.0006 | |
| 11 | GatedGCN | 0.9855
±0.0002 best reported | best at pub: Formatting Instructions for ICLR 2021 Conference Submissions (2020) |
| 32 | GIN | 0.9833
±0.0011 best reported | best at pub: Graph-Coupled Oscillator Networks (ICML 2022) |
| 86 | GAT | 0.9755
±0.0003 best reported | best at pub: Formatting Instructions for ICLR 2021 Conference Submissions (2020) |
| 95 | GraphSAGE | 0.9731
±0.0010 | This paper |
| 97 | SAGE | 0.9731
±0.0010 | |
| 126 | MLP | 0.9534
±0.0014 | This paper |
| 129 | 3WLGNN | 0.9507
±0.0096 | This paper |
| 135 | GCN | 0.9405
±0.0005 best reported | best at pub: A Generalization of ViT/MLP-Mixer to Graphs (ICML 2022) |
| 137 | MoNet | 0.9111 best reported | best at pub: Graph-Coupled Oscillator Networks (ICML 2022) |
| 139 | vanilla GCN | 0.9071
±0.0022 | This paper |
| 153 | RingGNN | 0.1135 | This paper |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | RandAlign | 0.7942
±0.0007 | |
| 2 | differential encoding | 0.7907
±0.0027 | |
| 3 | PPGT | 0.7856
±0.0070 | |
| 62 | GatedGCN | 0.7133
±0.0039 best reported | best at pub: Formatting Instructions for ICLR 2021 Conference Submissions (2020) |
| 92 | GraphSAGE | 0.6690
±0.0020 best reported | best at pub: Beyond Low-pass Filtering: Graph Convolutional Networks with Automatic Filtering (2021) |
| 101 | GAT | 0.6611
±0.0098 best reported | best at pub: Formatting Instructions for ICLR 2021 Conference Submissions (2020) |
| 103 | MoNet | 0.6591
±0.0251 best reported | best at pub: SoGCN: Second-Order Graph Convolutional Networks (2021) |
| 104 | SAGE | 0.6577
±0.0031 | |
| 109 | GIN | 0.6510
±0.0050 best reported | best at pub: Beyond Low-pass Filtering: Graph Convolutional Networks with Automatic Filtering (2021) |
| 123 | 3WLGNN | 0.5917
±0.0159 | This paper |
| 124 | GCN | 0.5839
±0.0073 best reported | best at pub: From Stars to Subgraphs: Uplifting Any GNN with Local Structure Awareness (ICLR 2021) |
| 126 | MLP | 0.5634
±0.0018 | |
| 129 | vanilla GCN | 0.5571
±0.0038 | This paper |
| 140 | RingGNN | 0.1930
±0.1611 | This paper |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | GRPE-Small | 0.8159
±0.0019 | |
| 2 | TANGO-GPS | 0.8011
±0.0014 | |
| 3 | GRIT | 0.8003
±0.0028 | |
| 55 | GatedGCN-PE | 0.7608
±0.0020 | This paper |
| 54 | GatedGCN | 0.7608
±0.0020 | |
| 67 | GAT | 0.7059
±0.0045 best reported | best at pub: How Expressive are Transformers in Spectral Domain for Graphs? (TMLR 2022) |
| 70 | GCN | 0.6903
±0.0137 best reported | best at pub: How Expressive are Transformers in Spectral Domain for Graphs? (TMLR 2022) best today: Recurrent Distance Filtering for Graph Representation Learning (ICML 2023) |
| 76 | MoNet | 0.6641
±0.0054 best reported | best at pub: SoGCN: Second-Order Graph Convolutional Networks (2021) |
| 79 | GIN | 0.6472
±0.0155 best reported | best at pub: How Expressive are Transformers in Spectral Domain for Graphs? (TMLR 2022) |
| 80 | GraphSAGE | 0.6388
±0.0011 best reported | best at pub: SoGCN: Second-Order Graph Convolutional Networks (2021) best today: IJCAI--23 Formatting Instructions (IJCAI 2023) |
| 85 | 3WLGNN | 0.6040
±0.0042 best reported | best at pub: SoGCN: Second-Order Graph Convolutional Networks (2021) Difference within reported uncertainty; not counted as underperformance. |
| 93 | vanilla GCN | 0.5344
±0.0203 | This paper |
| 104 | RingGNN | 0.4242
±0.2006 | This paper |
| 114 | MLP | 0.2097
±0.0000 | This paper |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | DeepGraph (48) | 0.9066
±0.0006 | |
| 2 | GRIT (our run) | 0.9041
±0.0023 | |
| 3 | GRIT 2-QiQRW | 0.9016
±0.0045 | |
| 67 | GatedGCN-PE | 0.8651
±0.0009 | This paper |
| 68 | GatedGCN (Rand sign(EigVecs)) | 0.8651
±0.0009 | This paper |
| 66 | GatedGCN | 0.8651
±0.0008 | |
| 69 | GatedGCN (Abs(EigVecs)) | 0.8639
±0.0004 | This paper |
| 72 | RingGNN | 0.8625
±0.0001 | This paper |
| 74 | GatedGCN (EigVecs-2) | 0.8603
±0.0009 | This paper |
| 78 | GatedGCN (Rand node ordering) | 0.8577
±0.0004 | This paper |
| 85 | 3WLGNN | 0.8566
±0.0035 | This paper |
| 88 | GCN | 0.8550
±0.0004 | prev best: How Expressive are Transformers in Spectral Domain for Graphs? (TMLR 2022) best today: Recurrent Distance Filtering for Graph Representation Learning (ICML 2023) |
| 89 | GIN | 0.8559
±0.0001 | This paper |
| 91 | MoNet | 0.8558
±0.0004 best reported | best at pub: SoGCN: Second-Order Graph Convolutional Networks (2021) |
| 107 | GAT | 0.7827
±0.0019 best reported | best at pub: How Expressive are Transformers in Spectral Domain for Graphs? (TMLR 2022) Difference within reported uncertainty; not counted as underperformance. |
| 119 | vanilla GCN | 0.6388
±0.0007 | This paper |
| 123 | MLP | 0.5052 | This paper |
| 124 | GraphSAGE | 0.5052
±0.0000 | This paper |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 8 | GCN | 1.0000 | prev best: A Generalization of ViT/MLP-Mixer to Graphs (ICML 2022) |
| 1 | PPGN | 1.0000 | |
| 2 | RW-AgentNet | 1.0000 | |
| 3 | AgentNet | 1.0000 | |
| 33 | MoNet | 0.9997
±0.0033 | This paper |
| 35 | GraphSAGE | 0.9993
±0.0047 | This paper |
| 34 | GAT | 0.9993
±0.0047 | This paper |
| 36 | GatedGCN | 0.9960
±0.0108 | prev best: A Generalization of ViT/MLP-Mixer to Graphs (ICML 2022) |
| 37 | GatedGCN (Rand sign(EigVecs)) | 0.9960
±0.0108 | This paper |
| 38 | GatedGCN (Abs(EigVecs)) | 0.9943
±0.0113 | This paper |
| 40 | GIN | 0.9933
±0.0133 | prev best: On the Equivalence between Graph Isomorphism Testing and Function Approximation with GNNs (NeurIPS 2019) |
| 46 | 3WLGNN | 0.9780
±0.1090 best reported | best at pub: Weisfeiler and Lehman Go Cellular: CW Networks (NeurIPS 2021) best today: Graph Self-Supervised Learning with Learnable Structural and Positional Encodings (WWW 2025) Difference within reported uncertainty; not counted as underperformance. |
| 69 | RingGNN | 0.1723
±0.0633 | prev best: Weisfeiler and Lehman Go Cellular: CW Networks (NeurIPS 2021) |
| 79 | All MP-GCNs | 0.1000 | This paper |
| # | Model | F1 ↑ | Paper |
|---|---|---|---|
| 1 | TGT-Agx4 | 0.8710
±0.0010 | |
| 2 | TGT-Agx3 | 0.8660
±0.0010 | |
| 3 | GIG-GatedGCN | 0.8630
±0.0010 | |
| 8 | GatedGCN | 0.8380
±0.0020 best reported | best at pub: Formatting Instructions for ICLR 2021 Conference Submissions (2020) best today: Graph in Graph Neural Network (2024) |
| 9 | GatedGCN-E | 0.8380
±0.0020 | This paper |
| 12 | RingGNN | 0.7040
±0.0030 | This paper |
| 15 | 3WLGNN | 0.6940
±0.0730 | This paper |
| 16 | k-NN Heuristic | 0.6930 | This paper |
| 18 | GAT | 0.6710
±0.0020 | This paper |
| 19 | GraphSAGE | 0.6650
±0.0030 | This paper |
| 20 | GIN | 0.6560
±0.0030 | This paper |
| 21 | GCN | 0.6430
±0.0010 | prev best: Formatting Instructions for ICLR 2021 Conference Submissions (2020) |
| 22 | MoNet | 0.6410
±0.0020 | This paper |
| 23 | vanilla GCN | 0.6300
±0.0010 | This paper |
| 24 | MLP | 0.5440
±0.0010 | This paper |
