Physics-informed generative model for drug-like molecule conformers

Summary

This paper proposes PIDM, a novel generative model for generating 3D molecular conformations from 2D molecular graphs. The proposed method uses an ODE based diffusion model to generate 3D molecular conformations, and designs a denoising network with multiple different modules to capture various of physical information from molecules. The proposed method achieves better performance than two baseline methods, GeoMol and GeoDiff, in experiments.

Strengths

- This work proposes a series of novel strategy to capture geometric information from different geometric inputs (e.g., bonds, bends, torsions) by different modules in a model. The proposed model architecture can be useful for a variety of molecular machine learning tasks. - Experimental results show that the proposed method achieves promising performance when compared with two strong baseline methods, GeoMol and GeoDiff, in benchmark datasets.

Weaknesses

Major: - A major weakness of this paper lies in that many necessary details of the proposed method is not presented or not sufficiently described as detailed below. (1) The proper torsions and improper torsions introduced in line 40-41 are not formally defined and it is not easy to discriminate them by Figure 1. Also, it is not clear why the model has a module for proper torsions but no module for improper torsions. (2) It is not easy to understand why "The constraint on proper torsions is cyclic in the angle $\phi$ such that energy is commonly parameterized as a function of $n\phi-\phi_0$" (line 46). Does it mean that proper torsions are periodic and $\phi$ is the periodicity? How this constraint is incorporated in the designed model? (3) The details of several modules are not described in Section 2. The details of the computation process in graph attention module is not described. The cis-trans module in Figure 2 is not introduced. Also, it is not clear what is the output ("solution" in Figure 2) of the model, is it the atom embedding, or the denoised atom coordinate? How the outputs are computed if they are not atom embeddings? - For experiments, it would make results stronger if authors can do comprehensive ablation studies about the proposed novel modules in the model as they are the major novelty contributions. Also, it is recommended to compare with an important baseline method, Torsional Diffusion [1]. Minor: - To make a better organization, it would be better to move Section 3 Datasets to the place just before Section 6 Experiments as they are more closely related. - It is hard to understand what Figure 4 tries to show. What do the y-axis values mean? And how these curves are related to explainability? Authors may make more clarification, and are recommended to move these results to experiment part to maintain a better organization. [1] Torsional Diffusion for Molecular Conformer Generation. NeurIPS 2022.

Questions

No additional questions.

Rating

3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

Yes.

Summary

The paper presents a diffusion model to generate conformers of drug-like molecules. The score model architecture is novel.

Strengths

The authors construct a diffusion model for conformer generation whose score model architecture is inspired by the structure of classical force fields. This is an interesting approach and quite different from the SE(3) or E(3) equivariant graph neural nets that are popular for this kind of problem.

Weaknesses

I found parts of the paper to be quite unclear. Line 46: I don’t understand this $n\phi - \phi_0$ expression or ‘freedom to select one of $n$ torsion angles’. Is there a discrete set of admissible torsion angles? Sorry, I’m ignorant about molecular geometry – but so will be many of your readers at NeurIPS. Line 71 ‘attempts to model nonbonded distances which requires it to sample torsional space during training… torsional space is physically ambiguous’: I don’t understand these sentences. Figure 2: what shape is the output from each of the pink blocks? Is it (number of atoms) * 3? What is the regression target for the summed ‘solution’? In some works on molecule generation or conformation prediction, the regression target is the noise that was added to the atom positions, and in other works the regression target is the ‘clean’ atom positions from the training data, so this needs to be clarified. Line 125: what is ‘conformer inconsistency at the graph level’? Section 4 Training: if you want to experiment with different samplers after training then I think that during training you should sample values of $\sigma$ from a continuous range, rather than picking from N discrete values. If you sample by stepping through values of $\sigma$ not seen during training, you will be presenting the score model with inputs it never saw during training, and I expect the performance will be suboptimal. Figure 4: please explain what the x and y axes are here. In general, I did not understand the ‘explainability’ aspect of the work. Line 216: I think that the inability of GeoDiff to discriminate between a molecule and its mirror image is a fundamental issue with the GFN architecture. Table 1 caption says 'best values in each category are highlighted' but I cannot see the highlight (perhaps a printing problem). The guided generation task in section 7 seems arbitrary. I do not understand equation (12) because in the right-hand-side numerator we have a vector $y$ and then subtract $F(y)$ which appears to be a scalar.

Questions

Line 95 ‘uses bonds as graph edges’: why not allow some message-passing between non-bonded atoms? Line 127: is any clustering used to split the data, or can similar molecules appear in both training and test data? Could you please elaborate on the explainability of the model? For example, show examples of predictions and their explanations. Perhaps this is meant to be shown in Figure 4 but I did not understand it.

Rating

3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

1 poor

Contribution

2 fair

Limitations

Yes

Summary

A diffusion-based conformation generation method called PIDM is proposed for molecules. It combines several geometries including bond lengths, bend angles, proper torsions, chirality and cis-trans of the noisy conformer as the input, and output the scores of the probability to iteratively generate the conformations. Experiments shows the robustness of the model, while is not convincing enough to show the performance of PIDM.

Strengths

- The proposed model is simple yet effective, with a smaller NFE used compared to GeoDiff. - Many version of the proposed DIPM has been test in the experiments.

Weaknesses

Several problems exist, so that I think it is not a mature enough article to be published. 1. Confusing presentation and limited novelty. - First, it is a physics-informed method, but I cannot find supports that there are some physics knowledge as priori knowledge that works in the model. The five geometries are very commonly used that conformation generation methods focus on, such as torsion angle in TorsionalDiff[1] and distance in CGCF[2]. It is not a novel idea based on physics rule. In the guided generation, some of the physics-based or energy-liked terms is used to modified the generating process, but it is not the main contribution of the paper, but a simple experimental attempt. - Second, the structure of the article is not well-organized. The description of the proposed method is very short, and not completely explained. More details with formulas should be added. The dimension of embedding sizes and other experimental details should be included in the experiment parts. Description of datasets and preprocessing should also placed in the experiment sections. In Sec. 5, most of the details has been given in [3], and the generation process is just an implementation of it, so it can be presented in a brief way, since it is not the main contributions. However, it takes up an entire page, almost the same as the designed method which is your main contributions. - Third, several parts are not well explained. For example, in Figure 2, what is the red arrow means, and what is the black means. Figure 3 is the loss in the training procedure on training a validation set, but what are you try to tell the readers? If you want to show the robustness of the model, some of evidence about other models are not robust should be compared, since most training/validation loss plots seem to be the same like Figure 3. There are also several similar examples. 2. Symmetries are not considered. As a diffusion models, most methods will consider the SO(3) equivariance or E(3) equivariance of the denoisers, leading the probability model to satisfy $p(x) = p(\Pi x)$ in which $\Pi$ is a matrix representation of the certain group. However, in PIDM, the output is scores of 3D-positions each atoms, but the input is the invariant geometries like bond lengths and angles, so if the noisy positions are rotated, the geometries will be unchanged, and the scores will also not be rotated. The equivariance is not ensured. (I am not very certain about it, because the detailed of model is not fully described in Sec.2, and the input and output of the model is mostly based on my guesses.) 3. Incomplete experiments. - The experimental protocols and compared baselines are not convincing. As the new SOTA method, TorsionalDiff shows better experimental performance compared to GeoMol. Both of these two methods generated a conformer based on a prior conformer generated by RDKit. Therefore, at least TorsionalDiff should be included as a baseline, and also, the experimental protocols such as metrics like Recall Coverage/AMR and Precision Coverage/AMR. The GeoDiff is only trained on a very small part of Drugs, but the checkpoint is directly employed for comparison. Is it reasonable? Or it should be trained on the same training set and test on validation set? For GeoMol as another deep-learning-based models, should it be re-trained and tested with the same protocols to should the superiority of the PIDM? - The comparison on geometries of generated molecules and ground-truth molecules is interesting, but some detailed comparison should be added. For example, the distribution of C-C, C=C bond distance of baselines, PIDM, and ground truth, or other bend angles, torsion angles, etc., and the JS-diversity between the distributions. Only the MAD cannot fully demonstrate the superiority of the performance. - In the abstract, it says that the model is resistant to overfitting and explainable. In which part of the experiments gives demonstration of these two advantages of the models. Can PIDM trained on Drugs generalize well on QMugs? If it can, the two datasets consist of molecules of similar sizes (25 ~ 30 atoms), how about smaller molecules in QM9 or larger ones in ligands in PDBBind? How the model shows explainability? - The sampling method used is in [3], what is the performance gain it brings compared to the original score-matching sampling methods? If there is no improvements according to empirical comparison, why it is employed? As I know, TorsionalDiff also use a very small NFE during sampling, so in evaluating effectiveness, does PIDM work better? [1] Bowen Jing, et al, Torsional Diffusion for Molecular Conformer Generation, https://arxiv.org/pdf/2206.01729.pdf [2] Chence Shi, et al. Learning Gradient Fields for Molecular Conformation Generation, https://arxiv.org/pdf/2105.03902v1)%3C%22 [3] Tero Karras, et al. Elucidating the Design Space of Diffusion-Based Generative Models, https://arxiv.org/pdf/2206.00364.pdf

Questions

The questions are given in the weakness part. Please refer to the points that I am doubtful about.

Rating

3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

1 poor

Contribution

2 fair

Limitations

N. A.

Summary

This paper introduces a physics-informed generative framework targeted at the task of molecular conformation generation. The method is motivated by the formulation of classical force fields and such an idea is reflected via bond, bend, and proper torsion and properly injected into the model design. The experiments are conducted on GEOM-Drugs and QMugs datasets in terms of molecule conformation generation.

Strengths

1. The paper is well-motivated via the lens of classical force fields and the method contains reasonable integration of the idea. 2. The method is easy to follow with necessary details presented to help understand the entire pipeline of the generation framework.

Weaknesses

1. The experimental evaluations lack the necessary details and discussions, which brings concerns about the credibility of the experimental comparison with the baselines (see Q1 and Q3). 2. Important baselines are missing (see Q2). Minor: Some parts of the presentation of the paper are confusing and may require further checks and polishment, e.g., line 246. In figure 5, is it possible to include some conformations generated by the baselines to offer the readers a clearer qualitative comparison?

Questions

1. In this paper, the authors seem to simply use the open-sourced checkpoints of the baselines like GeoMol and GeoDiff to report the testing results. However, it is uncertain whether the entire training protocol is fair compared with the original setups in these baseline papers. More justifications should be addressed on this point to ensure that the experiments offer a fair comparison. 2. Important baselines, e.g., Torsional Diffusion [1], are missing and should be discussed and compared since they share a similar/relevant idea of leveraging physical priors of bond/bend/torsional angles. 3. Why do the evaluation protocol and metrics seem so different from previous works like GeoMol and GeoDiff? Is it possible to conduct experiments following the widely-adopted setup in order to make a convincing comparison? [1] Jing, Bowen, et al. "Torsional diffusion for molecular conformer generation." Advances in Neural Information Processing Systems 2022.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

The authors have satisfactorily discussed the limitations.

Follow-up Responses

I appreciate authors' hard work in clarifying their method contributions and addressing my concerns in the rebuttal. - I appreciate authors' clarification and explanation about method details and Figure 4. - For paper organization, I think it is not awkward to move dataset introduction to places after method design. Unless your training method is specially designed for some certain datasets, it is more logically smooth to tell people first what are your method and novelty contributions, then demonstrate the effectiveness of your contributions by experiments on your used datasets (here you may introduce your datasets). It is a common organization style of papers in machine learning community though may not be the habit in other research areas. - I appreciate authors' clarification in the major target of the proposed method in their rebuttal to all reviewers. To my understanding, the major problem to be solved by PIDM is to generating a high-quality initial molecular conformations by creating accurate static terms (bond lengths & angles), and experiments show that PIDM can achieve it in experiments (Table 1). As for variable terms (proper torsions), PIDM does not consider improve their generation accuracy as they may change in different scenarios. I agree that it is not suitable to compare PIDM with Torsional Diffusion (TD). However, I am not persuaded that generating high-quality initial molecular conformations is a really important task in practical applications, as RDKit can also achieve it and produce a good enough initial conformation, this is why TD starts from RDKit produced conformation and refine proper torsions. In my opinion, authors should conduct many more experiments to motivate the importance of generating accurate static terms in producing accurate initial molecular conformations. For instance, in molecular conformation generation problem, authors may show PIDM + TD is much better than original TD (i.e., RDKit + TD); in the problem of generating ligand conformations conditioned on the given ligand and protein pocket [1], authors may show that replacing the commonly used RDKit initialized molecular conformations by PIDM initialized ones leads to much better performance. I suggest to more clearly clarify the central target of PIDM (i.e., focusing on static terms generation) and differentiate PIDM from other molecular conformation generation methods in the revision of paper. Also, more experiments should be added to demonstrate the importance of the studied problem. As these are not easily completed in a short periodic of author-reviewer discussion, I tend to keep my decision of rejection, but encourage authors to make significant reorganization in this paper following my suggestions and resubmit it to a new venue. [1] EQUIBIND: Geometric Deep Learning for Drug Binding Structure Prediction. ICML 2022.