Equivariant Blurring Diffusion for Hierarchical Molecular Conformer Generation

Summary

In this work, the authors introduce an hierarchical diffusion model for molecular conformer generation. The framework starts with fragment positions initialized by RDKit and designs a diffusion process that generates atomic positions from substructure positions. The reverse diffusion process is modeled by an equivariant neural network. Experiments on standard molecular conformer generation benchmarks: GEOM-QM9 and GEOM-DRUGS show better performance than some previous baselines. The authors also include ablation studies that validate some important design choices in the model.

Strengths

1. The model investigates a popular yet important problem: molecular conformer generation with deep generative model. 2. The proposed model is benchmarked on standard dataset to demonstrate its performance.

Weaknesses

1. The model is introduced in a way that it is derivated from blurring diffusion. However, the final formulation doesn't seem to strongly relate the concept. 2. The work misses comparison with some strong baselines in molecular conformer generation. 3. Though there are ablation studies to validate some design choices, there are still some important framework designs that are not well discussed. More details can be found in the following Questions section.

Questions

1. Following #1 in weakness, the final blurring operator (eq. 7) is basically a linear interpolant in Euclidean space between the substructure coordinate space and atom coordinate space. Unlike usual linear interpolant that starts from random Gaussian, here it starts from substructure coordinate space. I feel it is a bit confusing to introduce the framework as a variant of blurring diffusion which can obscure the actual contributions of this paper. 2. Following # 2 in weakness, there are recent deep generative models for molecular conformation generation [1,2] that achieve state-of-the-art but are not included in the comparison to proposed method. The authors are strongly recommended to include the comparison to better validation the performance. 3. Following # 3 in weakness, the work relies on principal subgraph (PS) to obtain molecular fragments. I wonder if the authors have tried other cheminformatic methods like BRICS [3]. 4. Also, the vocabulary size |S| is set to 50, which means there are quite some isolated atoms. I wonder what are the ratios of isolated atoms. 5. The work designs a diffusion process from substructure space to atom space, I wonder how it compare with a standard diffusion model that conditioned on substructure coordinates. Have the authors by any chance investigated similar settings? 5. How is $\delta^2$ in Eq. 8 determined in diffusion process? References: [1] Torsional Diffusion for Molecular Conformer Generation: https://arxiv.org/abs/2206.01729 [2] Swallowing the Bitter Pill: Simplified Scalable Conformer Generation: https://arxiv.org/abs/2311.17932 [3] On the Art of Compiling and Using 'Drug-Like' Chemical Fragment Spaces: https://chemistry-europe.onlinelibrary.wiley.com/doi/abs/10.1002/cmdc.200800178

Rating

5

Confidence

4

Soundness

2

Presentation

3

Contribution

2

Limitations

The authors adequately addressed the limitations in the work.

Summary

The paper proposes a novel diffusion method for molecular conformers based on blurring diffusion. The method utilizes RDKit to predict the 3D structure of small molecule fragments and trains a diffusion model to generate the full-atomistic molecule from the RDKit prior, leveraging hierarchical modeling. The method is evaluated based on the GEOM dataset, testing both the quality of the sampled geometries and physical properties of the conformers, as is standard in the field.

Strengths

1. The authors apply blurred diffusion developed for image applications to the domain of molecular conformer generation to leverage hierarchical modeling in molecular settings. 2. The problem of coarse-to-fine prediction is a core problem in coarse-grained molecular modeling (referred to as backmapping in the respective literature) and the proposed approach might be applicable these (large-scale) problems as well. 3. The proposed method demonstrates good performance compared to other diffusion-based approaches with models of comparable size in the literature.

Weaknesses

1. The authors did not consider the molecular conformer fields (MCF) paper (https://arxiv.org/pdf/2311.17932), which is the current state-of-the-art approach to molecular conformer generation. The MCF method is more performant than the proposed approach. 2. The approach depends strongly on coarse-grained fragment generation via RDKit, which could become a problem for larger systems of practical relevance. 3. The authors should consider also reporting their performance metrics on the stricter threshold on GEOM-Drugs (delta = 0.75 A).

Questions

1. How many parameters does the model use? 2. Did the authors try to build an end-to-end pipeline, where a generative model predicts the coarse-grained coordinates instead of RDKit?

Rating

4

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

The authors discuss the limitation of RDKit in application to larger molecular structure and the additional cost of the deblurring function.

Summary

The paper addresses the question by focusing on a fundamental biochemical problem: generating 3D molecular conformers based on molecular graphs in a multiscale manner. It consists of two stages: 1. Generating a coarse-grained fragment-level 3D structure from the molecular graph. 2. Generating fine atomic details from the coarse-grained approximated structure while allowing simultaneous adjustments to the latter.

Strengths

1. The paper proposed the EDB, which can generate atomic details from a coarse-to-grained estimation of fragment structures using equivariant networks 2. The paper proposed a novel blurring scheduler and a revised loss function that significantly impacts performance instead of directly applying those of the existing image blurring diffusion model. 3. The experiments and analysis demonstrate more plausible conformers compared to SOTA denoising diffusion models.

Weaknesses

None

Questions

None

Rating

5

Confidence

1

Soundness

3

Presentation

3

Contribution

3

Limitations

None

Summary

- This paper presents a model for small-molecule 3D structure generation, conditioning on its 2D molecular graphs. - The authors proposed a two-step process to address the problem: 1) first, using an off-the-shelf bioinformatics tool, RDKit, to generate a template scaffold structure; 2) then focusing on training a diffusion model to generate the fine-grained atom positions given the scaffold. - In essence, the model in the second step should learn to: 1) generate fine-grained atom positions given coarse fragment centers; 2) correct potential biases from RDKit-generated fragments. - To achieve this, they proposed a diffusion-like deblurring process inspired by heat diffusion (IHDM) but over a linear trajectory in Euclidean space, from fragment-averaged coordinates to predicted atomic coordinates. This design allows efficient training and sampling for the targeted problem. - Experiments on two small-molecule benchmarks show the model's superior performance compared to other generative model baselines.

Strengths

- The proposed model comprises a novel combination of rational design choices: 1. A scaffold-atomic two-step generation process that defers the first step to well-established tools, converting the problem to generate atomic details and correcting prior distribution. 2. Borrowing the idea from heat diffusion, it uses constant noise instead of varying noise levels as in regular diffusion models. 3. It uses updated trajectory matching objectives by matching $x_0$ instead of the next step $x_{t−1}$. - Empirical analysis shows these design choices bring noticeable improvement in sampling coverage and accuracy over previous diffusion models. They also present several ablation studies and analyses to understand the performance and pinpoint some design factors: 1) fragment size; 2) diffusion trajectory and noise schedules; 3) loss reparameterization. - The manuscript is presented in a clear manner and is easy to follow. - Overall, this paper demonstrates that certain design choices can lead to improved performance and can be valuable for further research in small-molecule generation tasks.

Weaknesses

- While the authors demonstrated better empirical performance and conducted ablation studies to verify selected design factors, some questions still remain on why and how some of the factors are critical, particularly: - The effect of using RDKit as prior distribution: see Q1 - Q2 - Experiment details on comparing constant noising schedule (proposed) to regular diffusion (DecompDiff like): see Q3 - The effect of choosing noising levels: see Q4 - Minor Typos: page 16: Pseudo-code 1: label is not for training code but the RDKit conformer generator.

Questions

Q1. The superior performance of the proposed models might be due to 1) the accurate generation of fine-grained atomic positions and/or 2) correcting the biases from RDKit-generated scaffolds. However, which component plays a more important role is not clearly addressed. The authors showed in Section 5.2 that a small fragment size of $|S|=50$ achieves the best performance due to the decreased atomic-level details needing to be learned, raising a natural question of whether the main benefit was from correcting the prior biases. For example, can the model achieve similar or better performance without fragments (i.e., set $S = \\{\text{atoms}\\}$ and the model only learns an error correction trajectory from the RDKit-predicted structure)? Q2. Related to Q1, one may wonder to what extent the model's performance relies on the quality of RDKit-generated scaffolds. Despite the discussion in the limitations, can the authors provide more analysis on EBD’s performance vs. RDKit’s performance? Q3. As discussed in 5.2, Effects of Data Corruptions, DecompDiff is the most similar model except for the choice of the diffusion process. However, the comparison with DecompDiff was limited: 1) it was not included in the full benchmark (Table 1); 2) T=50 steps were used for DecompDiff compared to T>200 in their paper, which might lead to different performances; 3) it was not clear if the authors retrained DecompDiff following the same setup, given the original DecompDiff was proposed for a different task (pocket-conditioned ligand generation). Can the authors provide clarification on above concerns? Q4. Sampling noise ($\sigma$, $\delta$) is a key hyperparameter in the proposed diffusion process. Can the authors provide theoretical or empirical analysis on the choices of these two parameters? Are the results sensitive to their choices?

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors have included discussion on the limitation in Appendix F.

Summary

The paper introduces Equivariant Blurring Diffusion (EBD), a unique generative model for hierarchical molecular conformer creation. A coarse-to-fine production process is presented by the model, with an emphasis on producing fragment-level structures first and then honing them down to atomic details. The method guarantees equivariance of SE(3), which is necessary for molecular structures. Comparisons with the most recent models on drug-like chemicals reveal that EBD performs better in geometric and chemical evaluations, demonstrating its effectiveness.

Strengths

An important step forward is the two-step process of creating fragment-level structures and then honing in on atomic details. This hierarchical method fits in nicely with molecular structures' multiscale structure. In order to preserve the geometrical and physical integrity of molecular conformers throughout the generation process, the model guarantees SE(3) equivariance. The experimental results show that EBD can generate accurate and diversified molecular conformers with fewer diffusion steps than state-of-the-art models. Comprehensive ablation investigations and in-depth comparisons with current models are included in the study, which offers a comprehensive understanding of the design decisions and how they affect performance. The examination of the chemical properties, which includes HOMO-LUMO gaps and energy estimates, provides a great deal of value by demonstrating that EBD can produce stable and chemically realistic conformers.

Weaknesses

1. The implementation and computational resource requirements may become more complex due to the hierarchical structure and requirement for fragmentation. This might make the model less useful and accessible for wider applications. 2. RDKit is used extensively in the first generation of fragment coordinates. The quality of the initial fragment structures that RDKit provides could limit the model's performance. 3. The concept works well for drug-like molecules, but it hasn't been fully investigated how well it scales to larger and more complex molecular structures. The claims would be strengthened by additional validation using larger datasets or more complicated compounds. 4. The geometric (RMSD) and chemical properties are the main metrics used for evaluation. An evaluation of the model's capabilities that is more thorough might be obtained by incorporating further metrics pertaining to the novelty and variety of the generated conformers.

Questions

1. Have you looked into any other options except RDKit for creating initial fragment coordinates? To what extent does the quality of these initial coordinates affect EBD performance? 2. Could you elaborate on how EBD scales up to more complex molecules? Have you used these datasets for any preliminary experiments? 3. Although the conformers developed exhibit chemical plausibility, what is their performance in real-world scenarios like docking simulations or property prediction? Is it planned to validate the conformers that are generated in these kinds of real-world situations? 4. There is a brief mention of generation times and training. Could you elaborate on the amount of computing power needed to train EBD and produce conformers? In what way does this differ from the resources required for other cutting-edge models?

Rating

7

Confidence

4

Soundness

2

Presentation

2

Contribution

3

Limitations

NA

Response to rebuttal

I thank the authors for answering the questions and adding extra experiments (i.e., analysis to PS and comparison to DecompDiff). I have raised my score.

I thank the authors for the detailed answers to my questions.

NA