Molecular Topological Profile (MOLTOP) - Simple and Strong Baseline for Molecular Graph Classification

Jakub Adamczyk, Wojciech Czech

Main figure for Molecular Topological Profile (MOLTOP) - Simple and Strong Baseline for Molecular Graph Classification

Abstract

We revisit the effectiveness of topological descriptors for molecular graph classification and design a simple, yet strong baseline. We demonstrate that a simple approach to feature engineering - employing histogram aggregation of edge descriptors and one-hot encoding for atomic numbers and bond types - when combined with a Random Forest classifier, can establish a strong baseline for Graph Neural Networks (GNNs). The novel algorithm, Molecular Topological Profile (MOLTOP), integrates Edge Betweenness Centrality, Adjusted Rand Index and SCAN Structural Similarity score. This approach proves to be remarkably competitive when compared to modern GNNs, while also being simple, fast, low-variance and hyperparameter-free. Our approach is rigorously tested on MoleculeNet datasets using fair evaluation protocol provided by Open Graph Benchmark. We additionally show out-of-domain generation capabilities on peptide classification task from Long Range Graph Benchmark. The evaluations across eleven benchmark datasets reveal MOLTOP's strong discriminative capabilities, surpassing the $1$-WL test and even $3$-WL test for some classes of graphs. Our conclusion is that descriptor-based baselines, such as the one we propose, are still crucial for accurately assessing advancements in the GNN domain.

Proposed method: MOLTOP

Benchmark Results

Tested on 4 datasets 21 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

MoleculeNet

#ModelROC-AUC ↑Paper
1
ProtoMol
0.9140 ±0.0030
2
MMSG
0.9080 ±0.0050
3
MemGNN
0.9070
20
D-MPNN
actual #20 vs paper #162 142 places worse
0.8780 ±0.0320
best reported
actual #20 vs paper #162 142 places worse
paper 0.8090 ±0.0060
43
GIN
actual #43 vs paper #82 39 places worse
0.8646 ±0.0081
best reported
actual #43 vs paper #82 39 places worse
paper 0.8450 ±0.0070
paper 0.8450 ±0.0070 Δ at pub 0.0160 Δ today 0.0196
45
AttentiveFP
actual #45 vs paper #211 166 places worse
0.8630 ±0.0150
best reported
actual #45 vs paper #211 166 places worse
paper 0.7840 ±0.0220
62
GEM
0.8560 ±0.0110
This paper
64
GCN
actual #64 vs paper #123 59 places worse
0.8540 ±0.0110
best reported
actual #64 vs paper #123 59 places worse
paper 0.8230 ±0.0340

Difference within reported uncertainty; not counted as underperformance.

paper 0.8230 ±0.0340 Δ at pub 0.0310
82
GraphSAGE
actual #82 vs paper #164 82 places worse
0.8451
best reported
actual #82 vs paper #164 82 places worse
paper 0.8070 ±0.0090
paper 0.8070 ±0.0090 Δ at pub 0.0381
95
ECFP
0.8380 ±0.0040
This paper
100
JOAO
actual #100 vs paper #261 161 places worse
0.8340 ±0.0330
best reported
actual #100 vs paper #261 161 places worse
paper 0.7290 ±0.0200
paper 0.7290 ±0.0200 Δ at pub 0.1050
110
MOLTOP
0.8290 ±0.0020
This paper
118
GROVER
actual #118 vs paper #156 38 places worse
0.8260 ±0.0070
best reported
actual #118 vs paper #156 38 places worse
paper 0.8100 ±0.0140

Difference within reported uncertainty; not counted as underperformance.

paper 0.8100 ±0.0140 Δ at pub 0.0160 Δ today 0.0160
149
GraphMVP
actual #149 vs paper #232 83 places worse
0.8120 ±0.0090
best reported
actual #149 vs paper #232 83 places worse
paper 0.7680 ±0.0110
paper 0.7680 ±0.0110 Δ at pub 0.0440 Δ today 0.0440
150
GraphMVP-C
0.8120 ±0.0090
This paper
166
LTP
0.8070 ±0.0030
This paper
169
LDP
actual #169 vs paper #169
0.8050 ±0.0030
actual #169 vs paper #169
paper 0.8050 ±0.0030

Improvement within reported uncertainty; not highlighted.

paper 0.8050 ±0.0030 Δ at pub 0.0034
213
GraphCL
actual #213 vs paper #286 73 places worse
0.7820 ±0.0120
best reported
actual #213 vs paper #286 73 places worse
paper 0.6870 ±0.0780

Difference within reported uncertainty; not counted as underperformance.

paper 0.6870 ±0.0780 Δ at pub 0.0950
273
molecular fingerprint
0.7150 ±0.0020
This paper
#ModelROC-AUC ↑Paper
1
MAGPrompt+
0.9432 ±0.0238
2
CamS-LLaMA
0.9420
3
SE(3)-I MPPH
0.9400 ±0.0210
34
GIN
actual #34 vs paper #185 151 places worse
0.8875 ±0.0049
best reported
actual #34 vs paper #185 151 places worse
paper 0.6870 ±0.0130
36
GraphCL
actual #36 vs paper #219 183 places worse
0.8870 ±0.0190
best reported
actual #36 vs paper #219 183 places worse
paper 0.6750 ±0.0330
paper 0.6750 ±0.0330 Δ at pub 0.2120
37
GROVER
actual #37 vs paper #166 129 places worse
0.8870 ±0.0060
best reported
actual #37 vs paper #166 129 places worse
paper 0.6950 ±0.0010
paper 0.6950 ±0.0010 Δ at pub 0.1920
38
GCN
actual #38 vs paper #127 89 places worse
0.8810 ±0.0360
best reported
actual #38 vs paper #127 89 places worse
paper 0.7060 ±0.0160
paper 0.7060 ±0.0160 Δ at pub 0.1750
39
JOAO
actual #39 vs paper #253 214 places worse
0.8790 ±0.0200
best reported
actual #39 vs paper #253 214 places worse
paper 0.6600 ±0.0060
paper 0.6600 ±0.0060 Δ at pub 0.2190
49
GraphMVP
actual #49 vs paper #190 141 places worse
0.8600 ±0.0340
best reported
actual #49 vs paper #190 141 places worse
paper 0.6850 ±0.0020
paper 0.6850 ±0.0020 Δ at pub 0.1750
57
GraphSAGE
actual #57 vs paper #164 107 places worse
0.6960 ±0.0190
actual #57 vs paper #164 107 places worse
paper 0.6960 ±0.0190
94
GraphMVP-C
0.7240 ±0.0160
This paper
93
GEM
0.7240 ±0.0040
This paper
183
MOLTOP
0.6890 ±0.0020
This paper
189
ECFP
0.6860 ±0.0050
This paper
194
Adding atoms and bonds features
0.6840 ±0.0060
This paper
263
LTP
0.6560 ±0.0030
This paper
274
Adding topological edge features
0.6500 ±0.0050
This paper
289
LDP
actual #289 vs paper #289
0.6330 ±0.0040
actual #289 vs paper #289
paper 0.6330 ±0.0040

Improvement within reported uncertainty; not highlighted.

paper 0.6330 ±0.0040 Δ at pub 0.0053
290
LDP, 50 bins
0.6330 ±0.0040
This paper
#ModelROC-AUC ↑Paper
1
GeomGCL
0.7630
2
LiGhT DeepGPT
0.7570 ±0.0110
3
CoMPT
0.7550
8
AttentiveFP
actual #8 vs paper #135 127 places worse
0.7430
best reported
actual #8 vs paper #135 127 places worse
paper 0.6370 ±0.0020
paper 0.6370 ±0.0020 Δ at pub 0.1060
14
GCN
actual #14 vs paper #101 87 places worse
0.7350 ±0.0020
best reported
actual #14 vs paper #101 87 places worse
paper 0.6530 ±0.0010
paper 0.6530 ±0.0010 Δ at pub 0.0820
31
JOAO
actual #31 vs paper #168 137 places worse
0.7110 ±0.0140
best reported
actual #31 vs paper #168 137 places worse
paper 0.6270 ±0.0060
paper 0.6270 ±0.0060 Δ at pub 0.0840
32
GIN
actual #32 vs paper #94 62 places worse
0.6570 ±0.0060
actual #32 vs paper #94 62 places worse
paper 0.6570 ±0.0060
34
GROVER
actual #34 vs paper #101 67 places worse
0.7080 ±0.0150
best reported
actual #34 vs paper #101 67 places worse
paper 0.6530 ±0.0050
paper 0.6530 ±0.0050 Δ at pub 0.0550
42
GraphCL
actual #42 vs paper #167 125 places worse
0.6960 ±0.0230
best reported
actual #42 vs paper #167 125 places worse
paper 0.6280 ±0.0020
paper 0.6280 ±0.0020 Δ at pub 0.0680
44
GEM
0.6920 ±0.0040
This paper
50
GraphMVP
actual #50 vs paper #168 118 places worse
0.6890 ±0.0100
best reported
actual #50 vs paper #168 118 places worse
paper 0.6270 ±0.0010
paper 0.6270 ±0.0010 Δ at pub 0.0620
62
GraphSAGE
actual #62 vs paper #109 47 places worse
0.6490 ±0.0020
actual #62 vs paper #109 47 places worse
paper 0.6490 ±0.0020
99
D-MPNN
0.6550 ±0.0030
This paper
120
MOLTOP
0.6440 ±0.0030
This paper
140
Adding atoms and bonds features
0.6350 ±0.0030
This paper
162
GraphMVP-C
0.6310 ±0.0040
This paper
198
ECFP
0.6040 ±0.0040
This paper
199
Adding topological edge features
0.6030 ±0.0050
This paper
200
LTP
0.6000 ±0.0040
This paper
201
LDP
0.5950 ±0.0040
This paper
202
LDP, 50 bins
0.5950 ±0.0040
This paper
209
molecular fingerprint
0.5750 ±0.0010
This paper

LRGB

#ModelAP ↑Paper
1
S^2GCN
0.7311 ±0.0066
2
S4G
0.7293 ±0.0004
3
BuNN
0.7276 ±0.0065
4
GCN
actual #4 vs paper #142 138 places worse
0.7261 ±0.0011
best reported
actual #4 vs paper #142 138 places worse
paper 0.5930 ±0.0023
paper 0.5930 ±0.0023 Δ at pub 0.0930 Δ today 0.1331
26
GIN
0.7059 ±0.0089
34
GatedGCN
actual #34 vs paper #149 115 places worse
0.7006 ±0.0033
best reported
actual #34 vs paper #149 115 places worse
paper 0.5864 ±0.0077
paper 0.5864 ±0.0077 Δ at pub 0.0901 Δ today 0.1142
35
GCNII
actual #35 vs paper #159 124 places worse
0.6990 ±0.0030
best reported
actual #35 vs paper #159 124 places worse
paper 0.5543 ±0.0078
best at pub: LRGB (NeurIPS 2022)
paper 0.5543 ±0.0078 Δ today 0.1447
73
GatedGCN+RWSE
actual #73 vs paper #134 61 places worse
0.6685 ±0.0062
best reported
actual #73 vs paper #134 61 places worse
paper 0.6069 ±0.0035
paper 0.6069 ±0.0035 Δ at pub 0.0616
78
GINE
actual #78 vs paper #162 84 places worse
0.6621 ±0.0067
best reported
actual #78 vs paper #162 84 places worse
paper 0.5498 ±0.0079
paper 0.5498 ±0.0079 Δ at pub 0.1123 Δ today 0.1123
89
SAN+RWSE
actual #89 vs paper #111 22 places worse
0.6562 ±0.0075
best reported
actual #89 vs paper #111 22 places worse
paper 0.6439 ±0.0075

Difference within reported uncertainty; not counted as underperformance.

paper 0.6439 ±0.0075 Δ at pub 0.0123
109
MOLTOP
0.6459 ±0.0005
This paper
116
SAN+LapPE
0.6384 ±0.0121
This paper
118
Transformer+LapPE
0.6326 ±0.0126
This paper
181
MLP
0.4060 ±0.0021
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Improved at pub — later paper surpassed this result Best today — OK at publication; today's leaderboard best is elsewhere