Improving Molecular Pretraining with Complementary Featurizations

Yanqiao Zhu, Dingshuo Chen, Yuanqi Du, Yingze Wang, Q. Liu, Shu Wu

Main figure for Improving Molecular Pretraining with Complementary Featurizations

Abstract

-0.5em Molecular pretraining, which learns molecular representations over massive unlabeled data, has become a prominent paradigm to solve a variety of tasks in computational chemistry and drug discovery. Recently, prosperous progress has been made in molecular pretraining with different molecular featurizations, including 1D SMILES strings, 2D graphs, and 3D geometries. However, the role of molecular featurizations with their corresponding neural architectures in molecular pretraining remains largely unexamined. In this paper, through two case studies---chirality classification and aromatic ring counting---we first demonstrate that different featurization techniques convey chemical information differently. In light of this observation, we propose a simple and effective lecular pretraining framework with mplementary featurizations (MOCO). MOCO comprehensively leverages multiple featurizations that complement each other and outperforms existing state-of-the-art models that solely relies on one or two featurizations on a wide range of molecular property prediction tasks.

Benchmark Results

Tested on 8 datasets 29 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

MoleculeNet

#ModelROC-AUC ↑Paper
1
HIMP
0.8740 ±0.0050
2
GINE w/ VN
0.8720 ±0.0030
3
NaiveGINe+ K=3 w/ VN
0.8700 ±0.0040
14
GCN
0.8400 ±0.0040
31
JOAO
actual #31 vs paper #247 216 places worse
0.8230 ±0.0220
best reported
actual #31 vs paper #247 216 places worse
paper 0.7440 ±0.0070
paper 0.7440 ±0.0070 Δ at pub 0.0790
46
ContextPred
actual #46 vs paper #279 233 places worse
0.8060 ±0.0120
best reported
actual #46 vs paper #279 233 places worse
paper 0.7330 ±0.0050
paper 0.7330 ±0.0050 Δ at pub 0.0730
47
GraphCL
actual #47 vs paper #231 184 places worse
0.8050 ±0.0170
best reported
actual #47 vs paper #231 184 places worse
paper 0.7500 ±0.0030
paper 0.7500 ±0.0030 Δ at pub 0.0550
61
GraphMVP
actual #61 vs paper #246 185 places worse
0.7990 ±0.0180
best reported
actual #61 vs paper #246 185 places worse
paper 0.7450 ±0.0040
paper 0.7450 ±0.0040 Δ at pub 0.0540
71
GraphLoG
actual #71 vs paper #282 211 places worse
0.7960 ±0.0250
best reported
actual #71 vs paper #282 211 places worse
paper 0.7300 ±0.0030
paper 0.7300 ±0.0030 Δ at pub 0.0660
81
InfoGraph
actual #81 vs paper #282 201 places worse
0.7930
best reported
actual #81 vs paper #282 201 places worse
paper 0.7300 ±0.0070
paper 0.7300 ±0.0070 Δ at pub 0.0630
135
GraphSAGE
actual #135 vs paper #246 111 places worse
0.7744 ±0.0038
best reported
actual #135 vs paper #246 111 places worse
paper 0.7450 ±0.0040
164
MOCO
0.7670 ±0.0040
This paper
180
GPT-GNN
actual #180 vs paper #211 31 places worse
0.7610 ±0.0040
best reported
actual #180 vs paper #211 31 places worse
paper 0.7530 ±0.0050
paper 0.7530 ±0.0050 Δ at pub 0.0080
185
---
0.7590 ±0.0030
This paper
202
GraphMAE
actual #202 vs paper #231 29 places worse
0.7550 ±0.0060
best reported
actual #202 vs paper #231 29 places worse
paper 0.7500 ±0.0040

Difference within reported uncertainty; not counted as underperformance.

paper 0.7500 ±0.0040 Δ at pub 0.0050 Δ today 0.0050
222
GROVER-C
0.7520 ±0.0030
This paper
229
AttrMask
actual #229 vs paper #259 30 places worse
0.7510 ±0.0090
best reported
actual #229 vs paper #259 30 places worse
paper 0.7420 ±0.0080

Difference within reported uncertainty; not counted as underperformance.

paper 0.7420 ±0.0080 Δ today 0.0090
281
GROVER-M
0.7320 ±0.0080
This paper
#ModelROC-AUC ↑Paper
1
ProtoMol
0.9140 ±0.0030
2
MMSG
0.9080 ±0.0050
3
MemGNN
0.9070
43
GIN
0.8646 ±0.0081
82
GraphSAGE
actual #82 vs paper #295 213 places worse
0.8451
best reported
actual #82 vs paper #295 213 places worse
paper 0.6460 ±0.0470
paper 0.6460 ±0.0470 Δ today 0.1991
98
GraphLoG
actual #98 vs paper #206 108 places worse
0.8350 ±0.0120
best reported
actual #98 vs paper #206 108 places worse
paper 0.7880 ±0.0070
paper 0.7880 ±0.0070 Δ at pub 0.0380 Δ today 0.0470
100
JOAO
actual #100 vs paper #261 161 places worse
0.8340 ±0.0330
best reported
actual #100 vs paper #261 161 places worse
paper 0.7290 ±0.0200
paper 0.7290 ±0.0200 Δ at pub 0.1050
105
GraphMAE
actual #105 vs paper #141 36 places worse
0.8310 ±0.0090
best reported
actual #105 vs paper #141 36 places worse
paper 0.8140 ±0.0140

Difference within reported uncertainty; not counted as underperformance.

paper 0.8140 ±0.0140 Δ at pub 0.0170 Δ today 0.0170
117
MOCO
0.8260 ±0.0030
This paper
149
GraphMVP
actual #149 vs paper #232 83 places worse
0.8120 ±0.0090
best reported
actual #149 vs paper #232 83 places worse
paper 0.7680 ±0.0110
paper 0.7680 ±0.0110 Δ at pub 0.0440 Δ today 0.0440
182
AttrMask
actual #182 vs paper #226 44 places worse
0.7970 ±0.0033
best reported
actual #182 vs paper #226 44 places worse
paper 0.7720 ±0.0140
186
ContextPred
actual #186 vs paper #207 21 places worse
0.7960 ±0.0120
best reported
actual #186 vs paper #207 21 places worse
paper 0.7860 ±0.0140

Difference within reported uncertainty; not counted as underperformance.

paper 0.7860 ±0.0140 Δ today 0.0100
197
GROVER-C
0.7920 ±0.0030
This paper
204
InfoGraph
actual #204 vs paper #255 51 places worse
0.7890 ±0.0110
best reported
actual #204 vs paper #255 51 places worse
paper 0.7390 ±0.0250
paper 0.7390 ±0.0250 Δ at pub 0.0500
208
GPT-GNN
actual #208 vs paper #225 17 places worse
0.7860 ±0.0290
best reported
actual #208 vs paper #225 17 places worse
paper 0.7760 ±0.0050

Difference within reported uncertainty; not counted as underperformance.

paper 0.7760 ±0.0050 Δ at pub 0.0090 Δ today 0.0100
213
GraphCL
actual #213 vs paper #286 73 places worse
0.7820 ±0.0120
best reported
actual #213 vs paper #286 73 places worse
paper 0.6870 ±0.0780

Difference within reported uncertainty; not counted as underperformance.

paper 0.6870 ±0.0780 Δ at pub 0.0950
260
GROVER-M
0.7340 ±0.0400
This paper
271
---
0.7150 ±0.0270
This paper
#ModelROC-AUC ↑Paper
1
MAGPrompt+
0.9432 ±0.0238
2
CamS-LLaMA
0.9420
3
SE(3)-I MPPH
0.9400 ±0.0210
34
GIN
0.8875 ±0.0049
36
GraphCL
actual #36 vs paper #219 183 places worse
0.8870 ±0.0190
best reported
actual #36 vs paper #219 183 places worse
paper 0.6750 ±0.0330
paper 0.6750 ±0.0330 Δ at pub 0.2120
39
JOAO
actual #39 vs paper #253 214 places worse
0.8790 ±0.0200
best reported
actual #39 vs paper #253 214 places worse
paper 0.6600 ±0.0060
paper 0.6600 ±0.0060 Δ at pub 0.2190
40
ContextPred
actual #40 vs paper #118 78 places worse
0.8770 ±0.0260
best reported
actual #40 vs paper #118 78 places worse
paper 0.7120 ±0.0090
paper 0.7120 ±0.0090 Δ at pub 0.1650
49
GraphMVP
actual #49 vs paper #190 141 places worse
0.8600 ±0.0340
best reported
actual #49 vs paper #190 141 places worse
paper 0.6850 ±0.0020
paper 0.6850 ±0.0020 Δ at pub 0.1750
54
GraphLoG
actual #54 vs paper #205 151 places worse
0.8460 ±0.0080
best reported
actual #54 vs paper #205 151 places worse
paper 0.6780 ±0.0170
paper 0.6780 ±0.0170 Δ at pub 0.1680
57
GraphSAGE
actual #57 vs paper #278 221 places worse
0.8304 ±0.0038
best reported
actual #57 vs paper #278 221 places worse
paper 0.6450 ±0.0310
101
GraphMAE
actual #101 vs paper #123 22 places worse
0.7200 ±0.0060
best reported
actual #101 vs paper #123 22 places worse
paper 0.7090 ±0.0090

Difference within reported uncertainty; not counted as underperformance.

paper 0.7090 ±0.0090 Δ at pub 0.0110 Δ today 0.0110
112
MOCO
0.7160 ±0.0100
This paper
122
---
0.7100 ±0.0050
This paper
138
GROVER-C
0.7030 ±0.0160
This paper
141
AttrMask
0.7020 ±0.0050
This paper
173
InfoGraph
0.6920 ±0.0080
This paper
220
GPT-GNN
actual #220 vs paper #278 58 places worse
0.6750 ±0.0130
best reported
actual #220 vs paper #278 58 places worse
paper 0.6450 ±0.0110
paper 0.6450 ±0.0110 Δ at pub 0.0300
246
GROVER-M
0.6640 ±0.0340
This paper
#ModelROC-AUC ↑Paper
1
SE(3)-I MPPH
0.9930 ±0.0040
2
KA-GCN
0.9920 ±0.0050
3
KA-GAT
0.9910 ±0.0050
19
GCN
0.9130 ±0.0170
40
AttrMask
actual #40 vs paper #183 143 places worse
0.8770 ±0.0119
best reported
actual #40 vs paper #183 143 places worse
paper 0.6860 ±0.0960
49
GraphMVP
actual #49 vs paper #96 47 places worse
0.8650 ±0.0170
best reported
actual #49 vs paper #96 47 places worse
paper 0.7900 ±0.0250
71
GraphMAE
actual #71 vs paper #77 6 places worse
0.8230 ±0.0120
best reported
actual #71 vs paper #77 6 places worse
paper 0.8150 ±0.0280

Difference within reported uncertainty; not counted as underperformance.

paper 0.8150 ±0.0280 Δ at pub 0.0080 Δ today 0.0080
77
MOCO
0.8160 ±0.0370
This paper
79
JOAO
actual #79 vs paper #195 116 places worse
0.8132 ±0.0249
best reported
actual #79 vs paper #195 116 places worse
paper 0.6630 ±0.0390
paper 0.6630 ±0.0390 Δ at pub 0.1500 Δ today 0.1502
97
GraphCL
0.7890 ±0.0420
This paper
103
GraphLoG
actual #103 vs paper #216 113 places worse
0.7872 ±0.0258
best reported
actual #103 vs paper #216 113 places worse
paper 0.6200 ±0.0180
paper 0.6200 ±0.0180 Δ at pub 0.1672 Δ today 0.1672
106
InfoGraph
actual #106 vs paper #128 22 places worse
0.7810
best reported
actual #106 vs paper #128 22 places worse
paper 0.7510 ±0.0500

Difference within reported uncertainty; not counted as underperformance.

paper 0.7510 ±0.0500 Δ at pub 0.0300
107
GROVER-M
0.7780 ±0.0200
This paper
115
ContextPred
actual #115 vs paper #142 27 places worse
0.7720 ±0.0088
best reported
actual #115 vs paper #142 27 places worse
paper 0.7370 ±0.0400

Difference within reported uncertainty; not counted as underperformance.

paper 0.7370 ±0.0400 Δ today 0.0350
131
GPT-GNN
actual #131 vs paper #232 101 places worse
0.7490 ±0.0270
best reported
actual #131 vs paper #232 101 places worse
paper 0.5780 ±0.0310
paper 0.5780 ±0.0310 Δ at pub 0.1710
165
---
0.7150 ±0.0530
This paper
223
GROVER-C
0.5990 ±0.0820
This paper
238
GraphSAGE
0.5580 ±0.0620
This paper
#ModelROC-AUC ↑Paper
1
KA-GAT
0.8470 ±0.0020
2
KA-GCN
0.8420 ±0.0010
3
GraphKAN
0.8370 ±0.0010
66
GIN
0.6403 ±0.0104
68
JOAO
actual #68 vs paper #150 82 places worse
0.6400 ±0.0100
best reported
actual #68 vs paper #150 82 places worse
paper 0.6070 ±0.0100
paper 0.6070 ±0.0100 Δ at pub 0.0330
70
GraphMVP
actual #70 vs paper #99 29 places worse
0.6390 ±0.0120
best reported
actual #70 vs paper #99 29 places worse
paper 0.6230 ±0.0160

Difference within reported uncertainty; not counted as underperformance.

paper 0.6230 ±0.0160 Δ at pub 0.0160 Δ today 0.0160
103
ContextPred
actual #103 vs paper #196 93 places worse
0.6220 ±0.0059
best reported
actual #103 vs paper #196 93 places worse
paper 0.5930 ±0.0140
114
GraphCL
actual #114 vs paper #178 64 places worse
0.6183 ±0.0060
best reported
actual #114 vs paper #178 64 places worse
paper 0.6010 ±0.0130

Difference within reported uncertainty; not counted as underperformance.

paper 0.6010 ±0.0130 Δ at pub 0.0070 Δ today 0.0173
123
GraphLoG
actual #123 vs paper #220 97 places worse
0.6150 ±0.0130
best reported
actual #123 vs paper #220 97 places worse
paper 0.5740 ±0.0230
paper 0.5740 ±0.0230 Δ at pub 0.0410
136
MOCO
0.6120 ±0.0060
This paper
134
AttrMask
actual #134 vs paper #169 35 places worse
0.6120 ±0.0012
best reported
actual #134 vs paper #169 35 places worse
paper 0.6040 ±0.0060

Difference within reported uncertainty; not counted as underperformance.

paper 0.6040 ±0.0060 Δ today 0.0080
160
GROVER-M
0.6060 ±0.0110
This paper
162
InfoGraph
actual #162 vs paper #199 37 places worse
0.6054 ±0.0090
best reported
actual #162 vs paper #199 37 places worse
paper 0.5920 ±0.0020

Difference within reported uncertainty; not counted as underperformance.

paper 0.5920 ±0.0020 Δ at pub 0.0020 Δ today 0.0134
175
GraphMAE
actual #175 vs paper #183 8 places worse
0.6030 ±0.0110
best reported
actual #175 vs paper #183 8 places worse
paper 0.5990 ±0.0050

Difference within reported uncertainty; not counted as underperformance.

paper 0.5990 ±0.0050 Δ at pub 0.0040 Δ today 0.0040
197
GPT-GNN
actual #197 vs paper #219 22 places worse
0.5930 ±0.0080
best reported
actual #197 vs paper #219 22 places worse
paper 0.5750 ±0.0420

Difference within reported uncertainty; not counted as underperformance.

paper 0.5750 ±0.0420 Δ at pub 0.0180
207
GROVER-C
0.5840 ±0.0060
This paper
218
---
0.5770 ±0.0310
This paper
227
GraphSAGE
0.5670 ±0.0010
This paper
#ModelROC-AUC ↑Paper
1
GeomGCL
0.7630
2
LiGhT DeepGPT
0.7570 ±0.0110
3
CoMPT
0.7550
14
GCN
0.7350 ±0.0020
28
ContextPred
actual #28 vs paper #167 139 places worse
0.7140 ±0.0180
best reported
actual #28 vs paper #167 139 places worse
paper 0.6280 ±0.0030
paper 0.6280 ±0.0030 Δ at pub 0.0860
31
JOAO
actual #31 vs paper #168 137 places worse
0.7110 ±0.0140
best reported
actual #31 vs paper #168 137 places worse
paper 0.6270 ±0.0060
paper 0.6270 ±0.0060 Δ at pub 0.0840
35
InfoGraph
actual #35 vs paper #184 149 places worse
0.7050
best reported
actual #35 vs paper #184 149 places worse
paper 0.6200 ±0.0030
paper 0.6200 ±0.0030 Δ at pub 0.0850
42
GraphCL
actual #42 vs paper #167 125 places worse
0.6960 ±0.0230
best reported
actual #42 vs paper #167 125 places worse
paper 0.6280 ±0.0020
paper 0.6280 ±0.0020 Δ at pub 0.0680
50
GraphMVP
actual #50 vs paper #168 118 places worse
0.6890 ±0.0100
best reported
actual #50 vs paper #168 118 places worse
paper 0.6270 ±0.0010
paper 0.6270 ±0.0010 Δ at pub 0.0620
62
GraphSAGE
actual #62 vs paper #194 132 places worse
0.6775 ±0.0041
best reported
actual #62 vs paper #194 132 places worse
paper 0.6080 ±0.0050
64
GraphLoG
actual #64 vs paper #181 117 places worse
0.6770 ±0.0190
best reported
actual #64 vs paper #181 117 places worse
paper 0.6220 ±0.0040
paper 0.6220 ±0.0040 Δ at pub 0.0550
110
MOCO
0.6490 ±0.0080
This paper
113
---
0.6470 ±0.0230
This paper
128
GraphMAE
0.6410 ±0.0010
This paper
150
AttrMask
actual #150 vs paper #177 27 places worse
0.6330 ±0.0060
best reported
actual #150 vs paper #177 27 places worse
paper 0.6250 ±0.0040

Difference within reported uncertainty; not counted as underperformance.

paper 0.6250 ±0.0040 Δ today 0.0080
151
GPT-GNN
actual #151 vs paper #181 30 places worse
0.6330 ±0.0030
best reported
actual #151 vs paper #181 30 places worse
paper 0.6220 ±0.0010
paper 0.6220 ±0.0010 Δ at pub 0.0090 Δ today 0.0110
174
GROVER-C
0.6260 ±0.0030
This paper
175
GROVER-M
0.6260 ±0.0050
This paper
#ModelRMSE ↓Paper
1
Graphormer-SPIS
0.4840 ±0.0050
2
Graphormer-SPD
0.4920 ±0.0040
3
conan-FGW
0.5140 ±0.0190
7
GCN
0.5640 ±0.0150
83
MOCO
0.9840 ±0.0340
This paper
98
GraphMVP
1.0910 ±0.0210
This paper
103
AttrMask
1.1120 ±0.0480
This paper
105
JOAO
1.1200 ±0.0190
This paper
114
ContextPred
1.1960 ±0.0370
This paper
123
---
1.3640 ±0.0160
This paper
#ModelRMSE ↓Paper
1
MemGNN
0.5560 ±0.0230
2
ProtoS-L2
0.5800 ±0.0160
3
ProtoW-L2
0.6040 ±0.0140
7
GCN
0.6550 ±0.0360
17
ContextPred
0.7020 ±0.0200
This paper
20
MOCO
0.7070 ±0.0010
This paper
21
JOAO
0.7080 ±0.0070
This paper
27
GraphMVP
0.7180 ±0.0160
This paper
31
AttrMask
0.7300 ±0.0040
This paper
37
---
0.7360 ±0.0060
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best today — OK at publication; today's leaderboard best is elsewhere