DiGRAF: Diffeomorphic Graph-Adaptive Activation Function

Krishna Sri Ipsit Mantri, Xinzhi Wang, C. Schönlieb, Bruno Ribeiro, Beatrice Bevilacqua, Moshe Eliasof

Abstract

In this paper, we propose a novel activation function tailored specifically for graph data in Graph Neural Networks (GNNs). Motivated by the need for graph-adaptive and flexible activation functions, we introduce DiGRAF, leveraging Continuous Piecewise-Affine Based (CPAB) transformations, which we augment with an additional GNN to learn a graph-adaptive diffeomorphic activation function in an end-to-end manner. In addition to its graph-adaptivity and flexibility, DiGRAF\ also possesses properties that are widely recognized as desirable for activation functions, such as differentiability, boundness within the domain, and computational efficiency. We conduct an extensive set of experiments across diverse datasets and tasks, demonstrating a consistent and superior performance of DiGRAF\ compared to traditional and graph-specific activation functions, highlighting its effectiveness as an activation function for GNNs.

Benchmark Results

Tested on 5 datasets

Tables show presented results in global context (best 3 papers + GCN + MLP).

TU Dortmund

MUTAG

Global leaderboard →

#	Model	Accuracy ↑	Paper
1	MSH-GNN GNN global	0.9910 ±0.0030	Multi-Scale Harmonic Encoding for Feature-Wise Graph Message Passing
2	ID-GNN+BEC Hyb global	0.9830 ±0.0360	Depth-Adaptive Graph Neural Networks via Learnable Bakry-Émery Curvature
3	SupCosine Hyb global	0.9830 ±0.0250	Supervised Contrastive Learning with Structure Inference for Graph Classification
39	GIN GNN global	0.9330 ±0.0290	ProtGNN: Towards Self-Explaining Graph Neural Networks
60	GIN + Tanh GNN baseline	0.9250 ±0.0790	This paper
71	GIN + DiGRAF GNN ★ proposed	0.9210 ±0.0790	This paper
74	GIN + Median GNN baseline	0.9200 ±0.0660	This paper
90	GIN + Maxout GNN baseline	0.9150 ±0.0750	This paper
98	GIN + Identity GNN baseline actual #98 vs paper #98	0.9140 ±0.0560	actual #98 vs paper #98 paper 0.9140 ±0.0560 prev best: GRANOLA: Adaptive Normalization for Graph Neural Networks (NeurIPS 2024) Improvement within reported uncertainty; not highlighted. paper 0.9140 ±0.0560 Δ at pub 0.0350
124	GIN + Sigmoid GNN baseline	0.9090 ±0.0550	This paper
125	GIN + LeakyReLU GNN baseline	0.9090 ±0.0570	This paper
126	GIN + GeLU GNN baseline	0.9090 ±0.0700	This paper
127	GIN + Max GNN baseline	0.9090 ±0.0710	This paper
163	GIN + Swish GNN baseline	0.9040 ±0.0480	This paper
242	GIN + ReLU GNN baseline	0.8940 ±0.0560	This paper
403	MLP code ↗ Trad global	0.8660 ±0.0495	Breaking the Limits of Message Passing Graph Neural Networks code ↗

NCI1

Global leaderboard →

#	Model	Accuracy ↑	Paper
1	WL-MLP GT global	0.9954	1-WL Expressiveness Is (Almost) All You Need
2	GIN GNN global	0.9934	1-WL Expressiveness Is (Almost) All You Need
3	GCN GNN global	0.9727	1-WL Expressiveness Is (Almost) All You Need
48	GIN + Median GNN baseline	0.8360 ±0.0190	This paper
51	GIN + LeakyReLU GNN baseline	0.8350 ±0.0220	This paper
57	GIN + DiGRAF GNN ★ proposed	0.8340 ±0.0120	This paper
56	GIN + Swish GNN baseline	0.8340 ±0.0140	This paper
60	GIN + GeLU GNN baseline	0.8330 ±0.0200	This paper
61	GIN + Maxout GNN baseline	0.8330 ±0.0290	This paper
62	GIN + Max GNN baseline	0.8330 ±0.0200	This paper
65	GIN + Tanh GNN baseline	0.8320 ±0.0260	This paper
84	GIN + Identity GNN baseline actual #84 vs paper #84	0.8280 ±0.0200	actual #84 vs paper #84 paper 0.8280 ±0.0200 prev best: GRANOLA: Adaptive Normalization for Graph Neural Networks (NeurIPS 2024) Improvement within reported uncertainty; not highlighted. paper 0.8280 ±0.0200 Δ at pub 0.0150
85	GIN + ReLU GNN baseline	0.8280 ±0.0250	This paper
98	GIN + Sigmoid GNN baseline	0.8260 ±0.0140	This paper
131	GraphSAGE code ↗ GNN global	0.8200 ±0.0100	Weisfeiler and Leman Go Infinite: Spectral and Combinatorial Pre-Colorings code ↗

PROTEINS

Global leaderboard →

#	Model	Accuracy ↑	Paper
1	MSH-GNN GNN global	0.9410 ±0.0330	Multi-Scale Harmonic Encoding for Feature-Wise Graph Message Passing
2	GRDL GNN global	0.8260 ±0.0120	Graph Classification via Reference Distribution Learning: Theory and Practice
3	GMN code ↗ Hyb global	0.8225	Memory-Based Graph Networks code ↗
32	GIN GNN global	0.7880 ±0.0410	Permutation-Invariant graph partitioning: How graph neural networks capture structural …
51	DiGRAF Hyb ★ proposed	0.7790 ±0.0340	This paper
91	GIN + Maxout GNN baseline	0.7680 ±0.0400	This paper
105	GIN + GeLU GNN baseline	0.7660 ±0.0280	This paper
136	GIN + ReLU GNN baseline	0.7620 ±0.0280	This paper
137	GIN + LeakyReLU GNN baseline	0.7620 ±0.0440	This paper
138	GIN + Swish GNN baseline	0.7620 ±0.0420	This paper
149	MLP Trad global	0.7600 ±0.0320	Simplifying Graph Convolutional Networks with Redundancy-Free Neighbors
157	GIN + Identity GNN baseline	0.7590 ±0.0320	This paper
158	GIN + Tanh GNN baseline	0.7590 ±0.0430	This paper
159	GIN + Max GNN baseline	0.7590 ±0.0310	This paper
249	GIN + Sigmoid GNN baseline	0.7500 ±0.0500	This paper
250	GIN + Median GNN baseline	0.7500 ±0.0430	This paper

OGB

ogbg-molhiv

Global leaderboard →

#	Model	ROC-AUC ↑	Paper
1	Multi-RF Fusion with Multi-GNN Blending global	0.8476 ±0.0002	Multi-RF Fusion with Multi-GNN Blending for Molecular Property Prediction
2	HyperFusion global	0.8475 ±0.0003	Multi-RF Fusion with Multi-GNN Blending for Molecular Property Prediction
3	PAS+FPs global	0.8420 ±0.0015	Multi-RF Fusion with Multi-GNN Blending for Molecular Property Prediction
9	GatedGCN GNN 2.8M global	0.8090 ±0.0160	Deep Prompt Tuning for Graph Transformers
21	GIN + DiGRAF Hyb ★ proposed	0.8028 ±0.0144	This paper
109	GIN GNN 510.0M global	0.7780 ±0.0182	GRPE: Relative Positional Encoding for Graph Transformer
222	GIN + ReLU GNN baseline	0.7558 ±0.0140	This paper
234	GIN + Tanh GNN baseline	0.7522 ±0.0203	This paper
236	GIN + Identity GNN baseline actual #236 vs paper #236	0.7512 ±0.0077	actual #236 vs paper #236 paper 0.7512 ±0.0077 prev best: GRANOLA: Adaptive Normalization for Graph Neural Networks (NeurIPS 2024) paper 0.7512 ±0.0077 Δ at pub 0.0323
244	GIN + LeakyReLU GNN baseline	0.7475 ±0.0120	This paper
256	GIN + GeLU GNN baseline	0.7415 ±0.0079	This paper
261	GIN + Sigmoid GNN baseline	0.7387 ±0.0080	This paper
266	GIN + Max GNN baseline	0.7344 ±0.0208	This paper
276	GIN + Swish GNN baseline	0.7295 ±0.0064	This paper
278	GIN + Median GNN baseline	0.7280 ±0.0221	This paper
279	GIN + Maxout GNN baseline	0.7275 ±0.0210	This paper

MoleculeNet

Tox21

Global leaderboard →

#	Model	ROC-AUC ↑	Paper
1	HIMP Hyb global	0.8740 ±0.0050	Graph convolutions that can finally model local structure
2	GINE w/ VN GNN global	0.8720 ±0.0030	Graph convolutions that can finally model local structure
3	NaiveGINe+ K=3 w/ VN GNN global	0.8700 ±0.0040	Graph convolutions that can finally model local structure
14	GCN GNN global	0.8400 ±0.0040	Graph convolutions that can finally model local structure
152	DiGRAF Hyb ★ proposed	0.7703 ±0.0059	This paper
203	GIN + Max GNN baseline	0.7550 ±0.0077	This paper
211	GIN + ReLU GNN baseline	0.7533 ±0.0051	This paper
227	GIN + Maxout GNN baseline	0.7514 ±0.0087	This paper
230	GIN + GeLU GNN baseline	0.7508 ±0.0062	This paper
234	GIN + Tanh GNN baseline	0.7493 ±0.0061	This paper
243	GIN + LeakyReLU GNN baseline	0.7460 ±0.0110	This paper
246	GIN + Identity GNN baseline	0.7451 ±0.0044	This paper
252	GIN + Median GNN baseline	0.7439 ±0.0090	This paper
279	GIN + Swish GNN baseline	0.7331 ±0.0101	This paper
309	GIN + Sigmoid GNN baseline	0.6915 ±0.0052	This paper

SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported value improved over the best known when published