Gradient Gating for Deep Multi-Rate Learning on Graphs
Abstract
We present Gradient Gating (G$^2$), a novel framework for improving the performance of Graph Neural Networks (GNNs). Our framework is based on gating the output of GNN layers with a mechanism for multi-rate flow of message passing information across nodes of the underlying graph. Local gradients are harnessed to further modulate message passing updates. Our framework flexibly allows one to use any basic GNN layer as a wrapper around which the multi-rate gradient gating mechanism is built. We rigorously prove that G$^2$ alleviates the oversmoothing problem and allows the design of deep GNNs. Empirical results are presented to demonstrate that the proposed framework achieves state-of-the-art performance on a variety of graph learning tasks, including on large-scale heterophilic graphs.
Benchmark Results
Tested on 7 datasets 21 baseline values reported too low
Tables show presented results in global context (best 3 papers + GCN + MLP).
Heterophilic Graphs
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | P^2GNN | 0.9672
±0.0147 | |
| 2 | DoG | 0.9660
±0.0210 | |
| 3 | GPF-plus | 0.9583
±0.0419 | |
| 28 | GPRGNN | 0.9295
±0.0131 best reported | best at pub: Restructuring Graphs for Higher Homophily via Adaptive Spectral Clustering (AAAI 2022) best today: GrokFormer: Graph Fourier Kolmogorov-Arnold Transformers (ICML 2024) |
| 34 | MLP | 0.9230
±0.0070 best reported | best at pub: Convolutional Neural Networks on Graphs with Chebyshev Approximation, Revisited (NeurIPS 2022) |
| 39 | GGCN | 0.9213
±0.0305 best reported | best at pub: Understanding convolution on graphs via energies (TMLR 2022) best today: Graph Wave Networks (WWW 2025) |
| 40 | H2GCN | 0.9189
±0.0393 best reported | best at pub: Graph-Coupled Oscillator Networks (ICML 2022) best today: GrokFormer: Graph Fourier Kolmogorov-Arnold Transformers (ICML 2024) Difference within reported uncertainty; not counted as underperformance. |
| 53 | GraphSAGE | 0.9000
±0.0365 best reported | best at pub: Graph Neural Networks Inspired by Classical Iterative Algorithms (ICML 2021) best today: Graph Neural Networks Need Cluster-Normalize-Activate Modules (NeurIPS 2024) Difference within reported uncertainty; not counted as underperformance. |
| 61 | FAGCN | 0.8961
±0.0152 best reported | best at pub: A Piece-wise Polynomial Filtering Approach for Graph Neural Networks (2021) best today: Node-oriented Spectral Filtering for Graph Neural Networks (2022) Difference within reported uncertainty; not counted as underperformance. |
| 70 | GCNII | 0.8918
±0.0443 best reported | best at pub: Convolutional Neural Networks on Graphs with Chebyshev Approximation, Revisited (NeurIPS 2022) best today: Piecewise Constant Spectral Graph Neural Network (TMLR 2025) |
| 99 | G^2-GraphSAGE | 0.8757
±0.0386 | This paper |
| 141 | GloGNN | 0.8524
±0.0490 best reported | best at pub: Finding Global Homophily in Graph Neural Networks When Meeting Heterophily (ICML 2022) best today: $P^2$GNN: Two Prototype Sets to boost GNN Performance (2026) Difference within reported uncertainty; not counted as underperformance. |
| 151 | G^2-GCN | 0.8486
±0.0324 | This paper |
| 161 | G^2-GAT | 0.8459
±0.0555 | This paper |
| 187 | F^2GAT | 0.8270
±0.0595 | This paper |
| 189 | SAGE | 0.8269 | |
| 211 | GAT | 0.8162
±0.0645 best reported | best at pub: Restructuring Graphs for Higher Homophily via Adaptive Spectral Clustering (AAAI 2022) best today: Equivariance Everywhere All At Once: A Recipe for Graph Foundation Models (2025) |
| 242 | ResGatedGCN | 0.8000
±0.0557 | This paper |
| 257 | GCN | 0.7933
±0.0447 best reported | best at pub: Restructuring Graphs for Higher Homophily via Adaptive Spectral Clustering (AAAI 2022) best today: Graph Wave Networks (WWW 2025) |
| 276 | MixHop | 0.7784
±0.0773 | This paper |
| 325 | LINKX | 0.7460
±0.0837 | This paper |
| 415 | Geom-GCN | 0.6760 best reported | best at pub: Non-Local Graph Neural Networks (2020) Difference within reported uncertainty; not counted as underperformance. |
| 499 | PairNorm | 0.6030
±0.0430 best reported | best at pub: Graph-Coupled Oscillator Networks (ICML 2022) Difference within reported uncertainty; not counted as underperformance. |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | P^2GNN | 0.9541
±0.0272 | |
| 2 | ACM-GCN | 0.9480
±0.0380 | |
| 3 | RF-GCN | 0.9429 | |
| 15 | GGCN | 0.9230
±0.0367 best reported | best at pub: Understanding convolution on graphs via energies (TMLR 2022) best today: $P^2$GNN: Two Prototype Sets to boost GNN Performance (2026) Difference within reported uncertainty; not counted as underperformance. |
| 22 | GPRGNN | 0.9157
±0.0196 best reported | best at pub: Comprehensive Graph Gradual Pruning for Sparse Training in Graph Neural Networks (2022) best today: Simplified PCNet with Robustness (2024) Difference within reported uncertainty; not counted as underperformance. |
| 24 | MLP | 0.9136
±0.0070 best reported | best at pub: Is Homophily a Necessity for Graph Neural Networks? (ICLR 2021) best today: Node-oriented Spectral Filtering for Graph Neural Networks (2022) Difference within reported uncertainty; not counted as underperformance. |
| 35 | GCNII | 0.8918
±0.0396 best reported | best at pub: Convolutional Neural Networks on Graphs with Chebyshev Approximation, Revisited (NeurIPS 2022) best today: $P^2$GNN: Two Prototype Sets to boost GNN Performance (2026) |
| 38 | FAGCN | 0.8852
±0.0133 best reported | best at pub: Comprehensive Graph Gradual Pruning for Sparse Training in Graph Neural Networks (2022) best today: Node-oriented Spectral Filtering for Graph Neural Networks (2022) Difference within reported uncertainty; not counted as underperformance. |
| 46 | G^2-GAT | 0.8730
±0.0484 | This paper |
| 61 | G^2-GCN | 0.8649
±0.0527 | This paper |
| 68 | H2GCN | 0.8623
±0.0471 best reported | best at pub: Graph-Coupled Oscillator Networks (ICML 2022) best today: $P^2$GNN: Two Prototype Sets to boost GNN Performance (2026) Difference within reported uncertainty; not counted as underperformance. |
| 70 | G^2-GraphSAGE | 0.8622
±0.0490 | This paper |
| 74 | GloGNN | 0.8595
±0.0510 best reported | best at pub: GPNet: Simplifying Graph Neural Networks via Multi-channel Geometric Polynomials (2022) best today: Aggregation-aware MLP: An Unsupervised Approach for Graph Message-passing (2025) Difference within reported uncertainty; not counted as underperformance. |
| 120 | F^2GAT | 0.8351
±0.0670 | This paper |
| 123 | LINKX | 0.8346
±0.0061 best reported | best at pub: GPNet: Simplifying Graph Neural Networks via Multi-channel Geometric Polynomials (2022) best today: Differentiable Cluster Graph Neural Network (2024) Difference within reported uncertainty; not counted as underperformance. |
| 138 | SAGE | 0.8269 | |
| 157 | GraphSAGE | 0.8196
±0.0739 best reported | best at pub: Graph-Coupled Oscillator Networks (ICML 2022) best today: Heterophily-Aware Graph Attention Network (2023) Difference within reported uncertainty; not counted as underperformance. |
| 158 | GAT | 0.8190
±0.0540 best reported | best at pub: Restructuring Graphs for Higher Homophily via Adaptive Spectral Clustering (AAAI 2022) |
| 170 | GCN | 0.8082
±0.0360 best reported | best at pub: Restructuring Graphs for Higher Homophily via Adaptive Spectral Clustering (AAAI 2022) best today: Bayesian Neighborhood Adaptation for Graph Neural Networks (TMLR 2025) |
| 183 | MixHop | 0.7950
±0.0630 best reported | best at pub: Label-Wise Graph Convolutional Network for Heterophilic Graphs (2021) Difference within reported uncertainty; not counted as underperformance. |
| 251 | ResGatedGCN | 0.7351
±0.0495 | This paper |
| 342 | Geom-GCN | 0.6412 best reported | best at pub: Graph Neural Networks Inspired by Classical Iterative Algorithms (ICML 2021) best today: Steering Graph Neural Networks with Pinning Control (2023) Difference within reported uncertainty; not counted as underperformance. |
| 389 | PairNorm | 0.5892
±0.0315 | This paper |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | DiGGR | 0.4535
±0.0353 | |
| 2 | TRIGON | 0.4402
±0.0028 | |
| 3 | DoG | 0.4320
±0.0120 | |
| 38 | GCNII | 0.4082
±0.0179 best reported | best at pub: Understanding convolution on graphs via energies (TMLR 2022) best today: $P^2$GNN: Two Prototype Sets to boost GNN Performance (2026) |
| 42 | MLP | 0.4026
±0.0108 best reported | best at pub: Convolutional Neural Networks on Graphs with Chebyshev Approximation, Revisited (NeurIPS 2022) best today: Heterophily-Aware Graph Attention Network (2023) |
| 52 | GPRGNN | 0.3992
±0.0067 best reported | best at pub: Comprehensive Graph Gradual Pruning for Sparse Training in Graph Neural Networks (2022) best today: GrokFormer: Graph Fourier Kolmogorov-Arnold Transformers (ICML 2024) |
| 56 | FAGCN | 0.3970 best reported | best at pub: Beyond Low-frequency Information in Graph Convolutional Networks (AAAI 2021) |
| 58 | GloGNN | 0.3959
±0.0029 best reported | |
| 62 | GGCN | 0.3929
±0.0048 best reported | best at pub: Understanding convolution on graphs via energies (TMLR 2022) best today: $P^2$GNN: Two Prototype Sets to boost GNN Performance (2026) Difference within reported uncertainty; not counted as underperformance. |
| 68 | GraphSAGE | 0.3899
±0.0085 best reported | best at pub: Deformable Graph Transformer (2022) best today: A Quasi-Wasserstein Loss for Learning Graph Neural Networks (WWW 2023) |
| 71 | H2GCN | 0.3885
±0.0117 best reported | best at pub: ES-GNN: Generalizing Graph Neural Networks Beyond Homophily with Edge Splitting (2022) best today: $P^2$GNN: Two Prototype Sets to boost GNN Performance (2026) |
| 87 | Geom-GCN | 0.3810 best reported | best at pub: Beyond Low-frequency Information in Graph Convolutional Networks (AAAI 2021) |
| 121 | G^2-GAT | 0.3730
±0.0087 | This paper |
| 127 | G^2-GraphSAGE | 0.3714
±0.0101 | This paper |
| 132 | G^2-GCN | 0.3709
±0.0116 | This paper |
| 135 | GCN | 0.3690 best reported | best at pub: Beyond Low-frequency Information in Graph Convolutional Networks (AAAI 2021) |
| 153 | F^2GAT | 0.3665
±0.0113 | This paper |
| 162 | MixHop | 0.3648
±0.0120 best reported | best at pub: Deformable Graph Transformer (2022) best today: Towards Deep Attention in Graph Neural Networks: Problems and Remedies (ICML 2023) |
| 180 | LINKX | 0.3610
±0.0155 | This paper |
| 182 | GAT | 0.3605
±0.0035 best reported | best at pub: Restructuring Graphs for Higher Homophily via Adaptive Spectral Clustering (AAAI 2022) best today: Hierarchical Graph Transformer with Adaptive Node Sampling (NeurIPS 2022) |
| 183 | ResGatedGCN | 0.3602
±0.0119 | This paper |
| 266 | GIN | 0.3410
±0.0030 | |
| 408 | PairNorm | 0.2740
±0.0124 | This paper |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | Hetero-S (GAT 8-layer) | 0.8693 | |
| 2 | Dir-GNN+CNA | 0.8586
±0.0180 | |
| 3 | GAT | 0.8560 best reported | best at pub: Restructuring Graphs for Higher Homophily via Adaptive Spectral Clustering (AAAI 2022) best today: The Heterophilic Snowflake Hypothesis: Training and Empowering GNNs for Heterophilic Graphs (KDD 2024) |
| 43 | G2-GraphSAGE | 0.7140
±0.0238 | This paper |
| 45 | GloGNN | 0.7133
±0.0242 best reported | best at pub: GPNet: Simplifying Graph Neural Networks via Multi-channel Geometric Polynomials (2022) best today: $P^2$GNN: Two Prototype Sets to boost GNN Performance (2026) Difference within reported uncertainty; not counted as underperformance. |
| 48 | GGCN | 0.7114
±0.0184 | This paper |
| 52 | GCN | 0.7033 best reported | best at pub: Non-Local Graph Neural Networks (2020) best today: Understanding Heterophily for Graph Neural Networks (ICML 2024) |
| 64 | LINKX | 0.6842
±0.0138 | This paper |
| 71 | F2GAT | 0.6781
±0.0205 | This paper |
| 72 | GPRGNN | 0.6748
±0.0040 best reported | best at pub: Simplifying approach to Node Classification in Graph Neural Networks (2021) best today: $P^2$GNN: Two Prototype Sets to boost GNN Performance (2026) |
| 99 | G2-GAT | 0.6412
±0.0196 | This paper |
| 103 | GCNII | 0.6386
±0.0304 | This paper |
| 107 | FAGCN | 0.6344
±0.0205 best reported | best at pub: Non-Local Graph Neural Networks (2020) |
| 108 | SAGE | 0.6340
±0.0300 | |
| 111 | PairNorm | 0.6274
±0.0282 | This paper |
| 115 | GraphSAGE | 0.6215
±0.0042 best reported | best at pub: Simplifying approach to Node Classification in Graph Neural Networks (2021) best today: Hierarchical Graph Transformer with Adaptive Node Sampling (NeurIPS 2022) |
| 124 | Geom-GCN | 0.6090 best reported | best at pub: Non-Local Graph Neural Networks (2020) best today: Beyond Smoothing: Unsupervised Graph Representation Learning with Edge Heterophily Discriminating (AAAI 2022) Difference within reported uncertainty; not counted as underperformance. |
| 129 | MixHop | 0.6050
±0.0253 | This paper |
| 133 | H2GCN | 0.6030
±0.0231 best reported | best at pub: Optimization-Induced Graph Implicit Nonlinear Diffusion (ICML 2022) Difference within reported uncertainty; not counted as underperformance. |
| 161 | G2-GCN | 0.5583
±0.0288 | This paper |
| 169 | MLP | 0.5429 best reported | best at pub: Block Modeling-Guided Graph Convolutional Neural Networks (AAAI 2021) best today: Understanding Heterophily for Graph Neural Networks (ICML 2024) |
| 199 | ResGatedGCN | 0.4982
±0.0271 | This paper |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | IBG-NN | 0.7763
±0.0179 | |
| 2 | Dir-GNN+CNA | 0.7747
±0.0128 | |
| 3 | DHGNN | 0.7684
±0.0161 | |
| 19 | GloGNN | 0.7121
±0.0184 best reported | best at pub: GPNet: Simplifying Graph Neural Networks via Multi-channel Geometric Polynomials (2022) best today: Directed Homophily-Aware Graph Neural Network (2025) |
| 30 | G2-GraphSAGE | 0.6426
±0.0238 | This paper |
| 39 | H2GCN | 0.6190
±0.0140 best reported | best at pub: Optimization-Induced Graph Implicit Nonlinear Diffusion (ICML 2022) best today: Efficient Learning on Large Graphs using a Densifying Regularity Lemma (2025) |
| 40 | LINKX | 0.6181
±0.0180 | This paper |
| 46 | GCN | 0.6067 best reported | best at pub: Restructuring Graphs for Higher Homophily via Adaptive Spectral Clustering (AAAI 2022) best today: Understanding Heterophily for Graph Neural Networks (ICML 2024) |
| 67 | GAT | 0.5630
±0.0220 best reported | best at pub: Restructuring Graphs for Higher Homophily via Adaptive Spectral Clustering (AAAI 2022) |
| 69 | GGCN | 0.5567
±0.0187 best reported | best at pub: GPNet: Simplifying Graph Neural Networks via Multi-channel Geometric Polynomials (2022) best today: $P^2$GNN: Two Prototype Sets to boost GNN Performance (2026) Difference within reported uncertainty; not counted as underperformance. |
| 76 | GPRGNN | 0.5435
±0.0087 best reported | best at pub: Simplifying approach to Node Classification in Graph Neural Networks (2021) best today: Directed Homophily-Aware Graph Neural Network (2025) |
| 88 | PairNorm | 0.5044
±0.0204 | This paper |
| 100 | F2GAT | 0.4732
±0.0243 | This paper |
| 105 | G2-GAT | 0.4648
±0.0141 | This paper |
| 120 | SAGE | 0.4460
±0.0130 | |
| 127 | MixHop | 0.4380
±0.0148 | This paper |
| 128 | GraphSAGE | 0.4378
±0.0190 best reported | best at pub: Simplifying approach to Node Classification in Graph Neural Networks (2021) best today: UniGAP: A Universal and Adaptive Graph Upsampling Approach to Mitigate Over-Smoothing in Node Classification Tasks (2024) Difference within reported uncertainty; not counted as underperformance. |
| 139 | FAGCN | 0.4259
±0.0079 | Improvement within reported uncertainty; not highlighted. |
| 167 | G2-GCN | 0.3962
±0.0291 | This paper |
| 172 | MLP | 0.3930
±0.0079 best reported | best at pub: Block Modeling-Guided Graph Convolutional Neural Networks (AAAI 2021) |
| 176 | GCNII | 0.3881
±0.0197 best reported | best at pub: GPNet: Simplifying Graph Neural Networks via Multi-channel Geometric Polynomials (2022) best today: $P^2$GNN: Two Prototype Sets to boost GNN Performance (2026) Difference within reported uncertainty; not counted as underperformance. |
| 179 | Geom-GCN | 0.3815
±0.0092 | prev best: Simplifying approach to Node Classification in Graph Neural Networks (2021) Improvement within reported uncertainty; not highlighted. |
| 192 | ResGatedGCN | 0.3760
±0.0180 | This paper |
LINKX Benchmarks
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | LargeScaleNet | 0.6582
±0.0036 | |
| 2 | S² DirGCN | 0.6495
±0.0033 | |
| 3 | FaberNet | 0.6462
±0.0101 | |
| 7 | G^2-GraphSAGE | 0.6330
±0.0184 | This paper |
| 13 | LINKX | 0.5600
±0.0134 | This paper |
| 16 | GloGNN | 0.5480
±0.0030 best reported | best at pub: Label-Wise Graph Convolutional Network for Heterophilic Graphs (2021) Difference within reported uncertainty; not counted as underperformance. |
| 17 | LINK | 0.5397
±0.0018 | This paper |
| 19 | MixHop | 0.5181
±0.0017 | prev best: Label-Wise Graph Convolutional Network for Heterophilic Graphs (2021) |
| 22 | GAT | 0.4937
±0.0020 best reported | best today: Uplifting Message Passing Neural Network with Graph Original Information (2022) |
| 25 | ACM-GCN | 0.4841
±0.0030 best reported | best today: Uplifting Message Passing Neural Network with Graph Original Information (2022) |
| 26 | GCNII | 0.4721
±0.0028 | This paper |
| 31 | GCN | 0.4602
±0.0026 | This paper |
| 32 | GPR-GNN | 0.4597
±0.0026 best reported | best at pub: Convolutional Neural Networks on Graphs with Chebyshev Approximation, Revisited (NeurIPS 2022) |
| 53 | APPNP | 0.3815
±0.0026 | This paper |
| 56 | MLP | 0.3670
±0.0021 | This paper |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | FaberNet | 0.7510
±0.0003 | |
| 2 | LargeScaleNet | 0.7505
±0.0005 | |
| 3 | DirGNN | 0.7395
±0.0005 | |
| 7 | G^2-GraphSAGE | 0.6950
±0.0039 | This paper |
| 10 | GloGNN | 0.6209
±0.0027 | This paper |
| 11 | LINKX | 0.6195
±0.0012 | This paper |
| 12 | LINK | 0.6039
±0.0007 | This paper |
| 14 | ACM-GCN | 0.5514
±0.0016 | This paper |
| 15 | MixHop | 0.5216
±0.0009 | This paper |
| 16 | GCN | 0.5102
±0.0006 best reported | best at pub: Non-Local Graph Neural Networks (2020) best today: Scale-aware Message Passing For Graph Node Classification (2024) |
| 23 | GAT | 0.4540
±0.0040 best reported | best at pub: Non-Local Graph Neural Networks (2020) Difference within reported uncertainty; not counted as underperformance. |
| 28 | GPR-GNN | 0.4019
±0.0003 | This paper |
| 30 | GCNII | 0.3788
±0.0069 | This paper |
| 31 | APPNP | 0.3219
±0.0007 | This paper |
| 32 | MLP | 0.3134
±0.0005 | prev best: Non-Local Graph Neural Networks (2020) best today: Understanding Heterophily for Graph Neural Networks (ICML 2024) Improvement within reported uncertainty; not highlighted. |
