NeurIPS 2023

Symmetry-Informed Geometric Representation for Molecules, Proteins, and Crystalline Materials

Shengchao Liu, Weitao Du, Yanjing Li, Zhuoxinran Li, Zhiling Zheng, Chenru Duan, Zhiming Ma, O. Yaghi, Anima Anandkumar, C. Borgs, J. Chayes, Hongyu Guo, Jian Tang

Main figure for Symmetry-Informed Geometric Representation for Molecules, Proteins, and Crystalline Materials

Abstract

Artificial intelligence for scientific discovery has recently generated significant interest within the machine learning and scientific communities, particularly in the domains of chemistry, biology, and material discovery. For these scientific problems, molecules serve as the fundamental building blocks, and machine learning has emerged as a highly effective and powerful tool for modeling their geometric structures. Nevertheless, due to the rapidly evolving process of the field and the knowledge gap between science (, physics, chemistry, & biology) and machine learning communities, a benchmarking study on geometrical representation for such data has not been conducted. To address such an issue, in this paper, we first provide a unified view of the current symmetry-informed geometric methods, classifying them into three main categories: invariance, equivariance with spherical frame basis, and equivariance with vector frame basis. Then we propose a platform, coined Geom3D, which enables benchmarking the effectiveness of geometric strategies. Geom3D contains 16 advanced symmetry-informed geometric representation models and 14 geometric pretraining methods over 46 diverse datasets, including small molecules, proteins, and crystalline materials. We hope that Geom3D can, on the one hand, eliminate barriers for machine learning researchers interested in exploring scientific problems; and, on the other hand, provide valuable guidance for researchers in computational chemistry, structural biology, and materials science, aiding in the informed selection of representation techniques for specific applications. The source code is available on the GitHub repository. =-1

Benchmark Results

Tested on 1 dataset 3 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

Quantum Chemistry

MD17

Global leaderboard →

Default metric is MAE. The paper reports 10 values on MAE , 10 values on Energy and 10 values on Force .

#ModelMAE ↓ defaultEnergy ↓Force ↓Paper
1
QHNet
1.04e-05
——
2
QHNetV2
1.04e-05
——
3
PhiSNet
1.76e-05
——
7
NequIP
0.0009
best reported
8.3330
23.7690
Energy 8.3330
Force 23.7690
paper 0.9360
14
Allegro
0.0290
1.1380
3.4050
Energy 1.1380
Force 3.4050
paper 0.0290
15
EGNN
0.0047 ±0.0002
best reported
17.8920
3.0420
Energy 17.8920
Force 3.0420
paper 0.3150
paper 0.3150 Δ today 0.3103
21
GemNet-T
0.0070
0.6840
0.5580
Energy 0.6840
Force 0.5580
paper 0.0070
paper 0.0070 Δ at pub 0.2130
22
PaiNN
0.0100
27.6260
0.5720
Energy 27.6260
Force 0.5720
paper 0.0100
26
Equiformer
0.0200
0.3080
0.2860
Energy 0.3080
Force 0.2860
This paper
46
SchNet
0.0500
best reported
0.4750
1.2030
Energy 0.4750
Force 1.2030
paper 0.1110
52
SphereNet
0.0520
0.2440
0.5460
Energy 0.2440
Force 0.5460
paper 0.0520
64
DimeNet++
0.0640
best reported
4.1680
7.2120
Energy 4.1680
Force 7.2120
paper 0.3400
paper 0.3400 Δ at pub 0.2760 Δ today 0.2760
89
SEGNN
0.1300
17.7740
9.0030
Energy 17.7740
Force 9.0030
This paper
SOTA performance (not significantly different from the best) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best at pub — reported value improved over the best known when published Improved at pub — later paper surpassed this result Best today — OK at publication; today's leaderboard best is elsewhere