Graph Structure Prompt Learning: A Novel Methodology to Improve Performance of Graph Neural Networks
Abstract
Graph neural networks (GNNs) are widely applied in graph data modeling. However, existing GNNs are often trained in a task-driven manner that fails to fully capture the intrinsic nature of the graph structure, resulting in sub-optimal node and graph representations. To address this limitation, we propose a novel Graph structure Prompt Learning method (GPL) to enhance the training of GNNs, which is inspired by prompt mechanisms in natural language processing. GPL employs task-independent graph structure losses to encourage GNNs to learn intrinsic graph characteristics while simultaneously solving downstream tasks, producing higher-quality node and graph representations. In extensive experiments on eleven real-world datasets, after being trained by GPL, GNNs significantly outperform their original performance on node classification, graph classification, and edge prediction tasks (up to 10.28\
Benchmark Results
Tested on 8 datasets 25 baseline values reported too low
Tables show presented results in global context (best 3 papers + GCN + MLP).
Classic
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | CNA | 0.9575
±0.0058 | |
| 2 | IS-GIB | 0.9390
±0.0187 | |
| 3 | EERM | 0.9112
±0.0145 | |
| 44 | GCN | 0.8033
±0.0088 | |
| 350 | MLP | 0.7289
±0.0010 | |
| 603 | GPPT | 0.7031
±0.0039 | prev best: ProG: A Graph Prompt Learning Benchmark (NeurIPS 2024) |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | LGTL | 0.9518
±0.0011 | |
| 2 | MixHop (LO) | 0.9513
±0.0038 | |
| 3 | GraphSAGE (LO) | 0.9511
±0.0036 | |
| 18 | GCN | 0.9296
±0.0015 best reported | best at pub: Node Identifiers: Compact, Discrete Representations for Efficient Graph Learning (ICLR 2024) |
| 20 | GAT | 0.9233
±0.0018 best reported | best at pub: Tackling Over-Smoothing for General Graph Convolutional Networks (2020) |
| 108 | ASDGN+GPL | 0.8903
±0.0012 | This paper |
| 115 | ASDGN | 0.8884
±0.0078 | This paper |
| 116 | SAGE | 0.8881
±0.0015 | |
| 125 | GraphSAGE | 0.8864
±0.0030 best reported | best at pub: Non-backtracking Graph Neural Networks (TMLR 2023) |
| 139 | GCN+PL | 0.8834
±0.0021 | This paper |
| 158 | FusedGAT+GPL | 0.8806
±0.0021 | This paper |
| 178 | LightGCN+GPL | 0.8767
±0.0001 | This paper |
| 181 | ARMA+GPL | 0.8762
±0.0005 | This paper |
| 197 | ChebNet+GPL | 0.8722
±0.0003 | This paper |
| 199 | ChebNet | 0.8718
±0.0029 best reported | best at pub: Non-backtracking Graph Neural Networks (TMLR 2023) |
| 210 | MLP | 0.8690
±0.0074 | |
| 217 | ARMA | 0.8678
±0.0021 | prev best: Graph Neural Networks With Convolutional ARMA Filters (2019) |
| 218 | UniMP +GPL | 0.8673
±0.0004 | This paper |
| 221 | GraphSage+GPL | 0.8660
±0.0028 | This paper |
| 238 | UniMP | 0.8568
±0.0026 | This paper |
| 240 | GAT+GPL | 0.8562
±0.0035 | This paper |
| 249 | GPPT | 0.8528
±0.0012 | This paper |
| 288 | FusedGAT | 0.8365
±0.0013 | This paper |
| 316 | LightGCN | 0.8256
±0.0013 | This paper |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | GraphTARIF | 0.9703
±0.0019 | |
| 2 | GAT | 0.9669
±0.0014 best reported | |
| 3 | GAT+T | 0.9660
±0.0007 | |
| 7 | SAGE | 0.9643
±0.0027 | |
| 8 | GCN | 0.9632
±0.0008 best reported | |
| 38 | GAT+GPL | 0.9535
±0.0014 | This paper |
| 45 | FusedGAT+GPL | 0.9527
±0.0030 | This paper |
| 49 | UniMP +GPL | 0.9513
±0.0062 | This paper |
| 56 | GraphSAGE | 0.9508
±0.0026 best reported | best at pub: A Quasi-Wasserstein Loss for Learning Graph Neural Networks (WWW 2023) Difference within reported uncertainty; not counted as underperformance. |
| 65 | UniMP | 0.9490
±0.0037 | This paper |
| 73 | GraphSage+GPL | 0.9456
±0.0015 | This paper |
| 74 | ARMA+GPL | 0.9452
±0.0019 | This paper |
| 100 | GCN+PL | 0.9401
±0.0014 | This paper |
| 119 | ChebNet | 0.9377
±0.0032 best reported | best at pub: LON-GNN: Spectral GNNs with Learnable Orthonormal Basis (2023) Difference within reported uncertainty; not counted as underperformance. |
| 126 | ASDGN+GPL | 0.9369
±0.0026 | This paper |
| 128 | ChebNet+GPL | 0.9367
±0.0022 | This paper |
| 248 | GPPT | 0.9276
±0.0026 | This paper |
| 261 | FusedGAT | 0.9269
±0.0044 | This paper |
| 288 | LightGCN+GPL | 0.9248
±0.0002 | This paper |
| 323 | ASDGN | 0.9222
±0.0069 | This paper |
| 369 | LightGCN | 0.9158
±0.0005 | This paper |
| 390 | MLP | 0.9128
±0.0063 | |
| 449 | ARMA | 0.9000
±0.0495 | prev best: Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View (AAAI 2019) Improvement within reported uncertainty; not highlighted. |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | GraphTARIF | 0.9461
±0.0017 | |
| 2 | GCN | 0.9412
±0.0008 best reported | best at pub: Clarify Confused Nodes via Separated Learning (2023) |
| 3 | GAT | 0.9398
±0.0022 best reported | |
| 8 | SAGE | 0.9351
±0.0006 | |
| 41 | FusedGAT+GPL | 0.9161
±0.0031 | This paper |
| 46 | GraphSAGE | 0.9120
±0.0029 best reported | best at pub: Polynormer: Polynomial-Expressive Graph Transformer in Linear Time (ICLR 2024) best today: Can TabPFN Compete with GNNs for Node Classification via Graph Tabularization? (LoG 2025) |
| 51 | FusedGAT | 0.9105
±0.0067 | This paper |
| 61 | GAT+GPL | 0.9076
±0.0017 | This paper |
| 66 | GCN+PL | 0.9067
±0.0025 | This paper |
| 115 | UniMP +GPL | 0.8984
±0.0067 | This paper |
| 151 | UniMP | 0.8916
±0.0154 | This paper |
| 196 | ASDGN+GPL | 0.8831
±0.0010 | This paper |
| 215 | GPPT | 0.8783
±0.0028 | This paper |
| 216 | ChebNet | 0.8782
±0.0072 best reported | best at pub: Node-oriented Spectral Filtering for Graph Neural Networks (2022) |
| 275 | GraphSage+GPL | 0.8635
±0.0081 | This paper |
| 295 | ChebNet+GPL | 0.8573
±0.0074 | This paper |
| 312 | ARMA+GPL | 0.8507
±0.0010 | This paper |
| 320 | MLP | 0.8463 | |
| 339 | LightGCN+GPL | 0.8428
±0.0005 | This paper |
| 355 | LightGCN | 0.8388
±0.0001 | This paper |
| 356 | ARMA | 0.8383
±0.0058 | prev best: Adaptive Propagation Graph Convolutional Network (2020) |
| 372 | ASDGN | 0.8348
±0.0847 | This paper |
TU Dortmund
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | MSH-GNN | 0.9910
±0.0030 | |
| 2 | ID-GNN+BEC | 0.9830
±0.0360 | |
| 3 | SupCosine | 0.9830
±0.0250 | |
| 16 | SortPool+GPL | 0.9600
±0.0200 | This paper |
| 23 | GIN+GPL | 0.9500
±0.0447 | This paper |
| 39 | GIN | 0.9330
±0.0290 best reported | best at pub: ProtGNN: Towards Self-Explaining Graph Neural Networks (AAAI 2021) |
| 63 | GCN | 0.9220
±0.0440 | |
| 101 | TopKPool | 0.9130
±0.0070 best reported | best at pub: Mind the Label Shift of Augmentation-based Graph OOD Generalization (2023) |
| 190 | SortPool | 0.9000
±0.0224 | prev best: CogDL: A Comprehensive Library for Graph Deep Learning (WWW 2021) Improvement within reported uncertainty; not highlighted. |
| 235 | MEWISPool+GPL | 0.8947
±0.0333 | This paper |
| 239 | DiffPool | 0.8940
±0.0460 best reported | best at pub: Total Variation Graph Neural Networks (ICML 2022) best today: Graph Classification via Reference Distribution Learning: Theory and Practice (NeurIPS 2024) Difference within reported uncertainty; not counted as underperformance. |
| 277 | DiffPool+GPL | 0.8900
±0.0300 | This paper |
| 403 | MLP | 0.8660
±0.0495 | |
| 456 | MEWISPool | 0.8473
±0.0473 best reported | best at pub: Union Subgraph Neural Networks (AAAI 2023) Difference within reported uncertainty; not counted as underperformance. |
| 589 | EdgePool | 0.8101
±0.0082 best reported | best at pub: Embedding Graphs on Grassmann Manifold (2022) |
| 601 | EdgePool+GPL | 0.8050
±0.0150 | This paper |
| 602 | TopKPool+GPL | 0.8050
±0.0522 | This paper |
| 623 | SAGPool | 0.7986
±0.0236 best reported | best at pub: Embedding Graphs on Grassmann Manifold (2022) |
| 636 | ASAPool+GPL | 0.7950
±0.0350 | This paper |
| 687 | GPS+GPL | 0.7650
±0.0320 | This paper |
| 722 | ASAPool | 0.7430
±0.0770 best reported | best at pub: Graph Information Bottleneck for Subgraph Recognition (ICLR 2020) best today: Recognizing Predictive Substructures with Subgraph Information Bottleneck (2021) Difference within reported uncertainty; not counted as underperformance. |
| 742 | SAGPool+GPL | 0.7300
±0.0871 | This paper |
| 772 | GPS | 0.6650
±0.0808 | This paper |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | WL-MLP | 0.9954 | |
| 2 | GIN | 0.9934 best reported | best at pub: 1-WL Expressiveness Is (Almost) All You Need (2022) |
| 3 | GCN | 0.9727 | |
| 5 | DiffPool | 0.9210
±0.0260 best reported | best at pub: Where to Mask: Structure-Guided Masking for Graph Masked Autoencoders (IJCAI 2024) |
| 131 | GraphSAGE | 0.8200
±0.0100 | |
| 149 | GPS | 0.8151
±0.0172 best reported | best today: Graph Persistence goes Spectral (NeurIPS 2025) |
| 175 | GIN+GPL | 0.8117
±0.0107 | This paper |
| 259 | SAGPool | 0.7910
±0.0300 best reported | best at pub: The expressive power of pooling in Graph Neural Networks (NeurIPS 2023) |
| 267 | EdgePool | 0.7893
±0.0222 best reported | best at pub: Graph Parsing Networks (ICLR 2024) |
| 355 | TopKPool | 0.7706
±0.0090 best reported | best at pub: Embedding Graphs on Grassmann Manifold (2022) |
| 407 | SortPool+GPL | 0.7538
±0.0086 | This paper |
| 415 | SortPool | 0.7496
±0.0134 | prev best: CogDL: A Comprehensive Library for Graph Deep Learning (WWW 2021) Improvement within reported uncertainty; not highlighted. |
| 434 | MEWISPool | 0.7421
±0.0326 best reported | best at pub: Union Subgraph Neural Networks (AAAI 2023) |
| 446 | EdgePool+GPL | 0.7331
±0.0081 | This paper |
| 448 | DiffPool+GPL | 0.7314
±0.0131 | This paper |
| 449 | ASAPool | 0.7310
±0.0250 best reported | best at pub: The expressive power of pooling in Graph Neural Networks (NeurIPS 2023) |
| 458 | SAGPool+GPL | 0.7260
±0.0137 | This paper |
| 470 | TopKPool+GPL | 0.7114
±0.0237 | This paper |
| 510 | MEWISPool+GPL | 0.6743
±0.0600 | This paper |
| 514 | ASAPool+GPL | 0.6640
±0.0108 | This paper |
| 536 | GPS+GPL | 0.6323
±0.0784 | This paper |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | MSH-GNN | 0.9410
±0.0330 | |
| 2 | GRDL | 0.8260
±0.0120 | |
| 3 | GMN | 0.8225 | |
| 6 | DiffPool+GPL | 0.8196
±0.0286 | This paper |
| 7 | DiffPool | 0.8161
±0.0232 | prev best: Memory-Based Graph Networks (ICLR 2020) |
| 11 | GIN+GPL | 0.8098
±0.0208 | This paper |
| 32 | GIN | 0.7880
±0.0410 best reported | best at pub: Permutation-Invariant graph partitioning: How graph neural networks capture structural interactions? (2023) Difference within reported uncertainty; not counted as underperformance. |
| 43 | SAGPool+GPL | 0.7821
±0.0215 | This paper |
| 58 | TopKPool | 0.7768 best reported | best at pub: Memory-Based Graph Networks (ICLR 2020) |
| 67 | SAGE | 0.7726
±0.0228 | |
| 86 | TopKPool+GPL | 0.7696
±0.0078 | This paper |
| 97 | ASAPool+GPL | 0.7670
±0.0129 | This paper |
| 134 | SortPool+GPL | 0.7625
±0.0091 | This paper |
| 149 | MLP | 0.7600
±0.0320 | |
| 152 | SortPool | 0.7598
±0.0176 | prev best: Memory-Based Graph Networks (ICLR 2020) Improvement within reported uncertainty; not highlighted. |
| 154 | SAGPool | 0.7595
±0.0452 best reported | best at pub: Embedding Graphs on Grassmann Manifold (2022) Difference within reported uncertainty; not counted as underperformance. |
| 184 | EdgePool | 0.7560
±0.0240 best reported | best at pub: Embedding Graphs on Grassmann Manifold (2022) |
| 323 | EdgePool+GPL | 0.7384
±0.0133 | This paper |
| 397 | ASAPool | 0.7232
±0.0291 | prev best: Graph Information Bottleneck for Subgraph Recognition (ICLR 2020) Improvement within reported uncertainty; not highlighted. |
| 449 | MEWISPool+GPL | 0.7060
±0.0286 | This paper |
| 464 | MEWISPool | 0.6940
±0.0144 | prev best: Union Subgraph Neural Networks (AAAI 2023) Improvement within reported uncertainty; not highlighted. |
| 471 | GPS+GPL | 0.6785
±0.0715 | This paper |
| 486 | GPS | 0.6446
±0.0426 | This paper |
Heterogeneous Graph Benchmarks
| # | Model | Micro-F1 ↑ | Paper |
|---|---|---|---|
| 1 | LMSPS | 0.9566
±0.0020 | |
| 2 | PSHGCN | 0.9561
±0.0012 | |
| 3 | RpHGNN | 0.9555
±0.0029 | |
| 4 | FusedGAT+GPL | 0.9544
±0.0023 | This paper |
| 21 | GCN | 0.9460
±0.0031 best reported | best at pub: Graph Neural Networks with a Distribution of Parametrized Graphs (ICML 2023) |
| 23 | GAT | 0.9448
±0.0022 best reported | best at pub: Graph Neural Networks with a Distribution of Parametrized Graphs (ICML 2023) |
| 42 | FusedGAT | 0.9340
±0.0020 | This paper |
| 46 | GraphSAGE | 0.9306
±0.0047 best reported | best at pub: Efficient Heterogeneous Graph Learning via Random Projection (2023) |
| 88 | GIN | 0.8720 | |
| 91 | GCN+PL | 0.8680
±0.0001 | This paper |
| 92 | GAT+GPL | 0.8673
±0.0023 | This paper |
| 94 | ChebNet+GPL | 0.8633
±0.0002 | This paper |
| 95 | ARMA+GPL | 0.8625
±0.0006 | This paper |
| 96 | UniMP +GPL | 0.8622
±0.0018 | This paper |
| 97 | GraphSage+GPL | 0.8607
±0.0026 | This paper |
| 98 | ARMA | 0.8581
±0.0013 | This paper |
| 100 | ChebNet | 0.8575
±0.0012 | This paper |
| 101 | LightGCN+GPL | 0.8555
±0.0002 | This paper |
| 103 | UniMP | 0.8537
±0.0021 | This paper |
| 110 | LightGCN | 0.8444
±0.0004 | This paper |
| 126 | ASDGN+GPL | 0.8121
±0.0020 | This paper |
| 134 | ASDGN | 0.8039
±0.0028 | This paper |
| 176 | MLP | 0.6447
±0.0136 |
