MAGPrompt: Message-Adaptive Graph Prompt Tuning for Graph Neural Networks

Long D. Nguyen, Binh P. Nguyen

Main figure for MAGPrompt: Message-Adaptive Graph Prompt Tuning for Graph Neural Networks

Abstract

Pre-trained graph neural networks (GNNs) transfer well, but adapting them to downstream tasks remains challenging due to mismatches between pre-training objectives and task requirements. Graph prompt tuning offers a parameter-efficient alternative to fine-tuning, yet most methods only modify inputs or representations and leave message passing unchanged, limiting their ability to adapt neighborhood interactions. We propose message-adaptive graph prompt tuning, which injects learnable prompts into the message passing step to reweight incoming neighbor messages and add task-specific prompt vectors during message aggregation, while keeping the backbone GNN frozen. The approach is compatible with common GNN backbones and pre-training strategies, and applicable across downstream settings. Experiments on diverse node- and graph-level datasets show consistent gains over prior graph prompting methods in few-shot settings, while achieving performance competitive with fine-tuning in full-shot regimes.

Proposed method: MAGPrompt+

Benchmark Results

Tested on 7 datasets 3 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

#ModelAccuracy ↑Paper
1
CNA
0.9575 ±0.0058
2
IS-GIB
0.9390 ±0.0187
3
EERM
0.9112 ±0.0145
44
GCN
0.8033 ±0.0088
350
MLP
0.7289 ±0.0010
603
GPPT
actual #603 vs paper #874 271 places worse
0.7031 ±0.0039
best reported
actual #603 vs paper #874 271 places worse
paper 0.4581 ±0.0354
832
GPF-plus
actual #832 vs paper #881 49 places worse
0.5967 ±0.1187
best reported
actual #832 vs paper #881 49 places worse
paper 0.4460 ±0.0447
best at pub: ProG: A Graph Prompt Learning Benchmark (NeurIPS 2024)

Difference within reported uncertainty; not counted as underperformance.

paper 0.4460 ±0.0447 Δ at pub 0.1507
862
MAGPrompt+
0.5282 ±0.0418
This paper
866
EdgePrompt+
actual #866 vs paper #866
0.4971 ±0.0225
actual #866 vs paper #866
paper 0.4971 ±0.0225
paper 0.4971 ±0.0225 Δ at pub 0.1117
869
GPF
actual #869 vs paper #869
0.4902 ±0.0453
actual #869 vs paper #869
paper 0.4902 ±0.0453

Improvement within reported uncertainty; not highlighted.

paper 0.4902 ±0.0453 Δ at pub 0.0413
871
GraphPrompt
actual #871 vs paper #871
0.4707 ±0.0309
actual #871 vs paper #871
paper 0.4707 ±0.0309

Improvement within reported uncertainty; not highlighted.

paper 0.4707 ±0.0309 Δ at pub 0.0173
882
All-In-One
actual #882 vs paper #882
0.4436 ±0.0252
actual #882 vs paper #882
paper 0.4436 ±0.0252
prev best: ProG: A Graph Prompt Learning Benchmark (NeurIPS 2024)

Improvement within reported uncertainty; not highlighted.

paper 0.4436 ±0.0252 Δ at pub 0.0395
886
GraphTOP
0.4113 ±0.0159
This paper
887
Linear Probe
0.4045 ±0.0355
This paper

TU Dortmund

#ModelAccuracy ↑Paper
1
WL-MLP
0.9954
2
GIN
0.9934
3
GCN
0.9727
131
GraphSAGE
0.8200 ±0.0100
516
EdgePrompt+
0.6630 ±0.0254
This paper
531
GPF-plus
0.6456 ±0.0110
This paper
538
GraphPrompt
0.6319 ±0.0171
This paper
539
GPF
0.6266 ±0.0183
This paper
542
Linear Probe
0.6249 ±0.0195
This paper
560
All-In-One
0.5914 ±0.0212
This paper
#ModelAccuracy ↑Paper
1
GMN
0.7866
2
MemGNN
0.7550
3
TFGW SP
0.7510 ±0.0500
17
GatedGCN
0.7000 ±0.0494
26
GraphSAGE
0.6817 ±0.0545
124
MLP
0.5583 ±0.0352
205
MAGPrompt+
0.3719 ±0.0279
This paper

MoleculeNet

#ModelROC-AUC ↑Paper
1
MAGPrompt+
0.9432 ±0.0238
This paper
2
CamS-LLaMA
0.9420
3
SE(3)-I MPPH
0.9400 ±0.0210
6
GPF-plus
actual #6 vs paper #6
0.9319 ±0.0206
actual #6 vs paper #6
paper 0.9319 ±0.0206
paper 0.9319 ±0.0206 Δ at pub 0.2101
7
Fine-tune
0.9299 ±0.0371
This paper
8
EdgePrompt+
0.9279 ±0.0357
This paper
9
GPF
actual #9 vs paper #9
0.9273 ±0.0282
actual #9 vs paper #9
paper 0.9273 ±0.0282
paper 0.9273 ±0.0282 Δ at pub 0.2162
34
GIN
0.8875 ±0.0049
#ModelROC-AUC ↑Paper
1
SE(3)-I MPPH
0.9930 ±0.0040
2
KA-GCN
0.9920 ±0.0050
3
KA-GAT
0.9910 ±0.0050
19
GCN
0.9130 ±0.0170
119
GPF-plus
actual #119 vs paper #173 54 places worse
0.7617 ±0.0298
best reported
actual #119 vs paper #173 54 places worse
paper 0.7003 ±0.1456

Difference within reported uncertainty; not counted as underperformance.

paper 0.7003 ±0.1456 Δ at pub 0.0614
129
GPF
actual #129 vs paper #145 16 places worse
0.7506 ±0.0102
best reported
actual #129 vs paper #145 16 places worse
paper 0.7347 ±0.1074

Difference within reported uncertainty; not counted as underperformance.

paper 0.7347 ±0.1074 Δ at pub 0.0159
161
MAGPrompt+
0.7181 ±0.1158
This paper
189
EdgePrompt+
0.6791 ±0.1273
This paper
202
Fine-tune
0.6499 ±0.1310
This paper
#ModelROC-AUC ↑Paper
1
KA-GAT
0.8470 ±0.0020
2
KA-GCN
0.8420 ±0.0010
3
GraphKAN
0.8370 ±0.0010
8
GPF
actual #8 vs paper #175 167 places worse
0.6913 ±0.0116
best reported
actual #8 vs paper #175 167 places worse
paper 0.6028 ±0.0536
paper 0.6028 ±0.0536 Δ at pub 0.0885
11
GPF-plus
actual #11 vs paper #150 139 places worse
0.6865 ±0.0072
best reported
actual #11 vs paper #150 139 places worse
paper 0.6075 ±0.0472
paper 0.6075 ±0.0472 Δ at pub 0.0790
66
GIN
0.6403 ±0.0104
105
Fine-tune
0.6217 ±0.0427
This paper
107
MAGPrompt+
0.6201 ±0.0436
This paper
150
EdgePrompt+
0.6076 ±0.0495
This paper
180
DGI
actual #180 vs paper #180
0.6009 ±0.0582
actual #180 vs paper #180
paper 0.6009 ±0.0582

Improvement within reported uncertainty; not highlighted.

paper 0.6009 ±0.0582 Δ at pub 0.0169
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best at pub — reported value improved over the best known when published