SubGDiff: A Subgraph Diffusion Model to Improve Molecular Representation Learning

Summary

SubgDiff is introduced to improve molecular representation learning by integrating substructural information into the diffusion model framework. It offers three key technical contributions (subgraph prediction, expectation state, and k-step same subgraph diffusion) to enhance the network's understanding of molecular substructures. Experiments were carried out on several downstream tasks, particularly molecular force predictions.

Strengths

* Incorporate substructural information into diffusion model

Weaknesses

* The denoising process need better explanations. * Diffusion models excel at generating new samples. The application of SubgDiff to molecular property prediction/classification does not show its strengths. The state-of-the-art results are missed in Section 5.1. Section 5.2 doesn't compare generation results with the state-of-the-art. * Explain COV-R and MAT-R in section 5.2

Questions

* Does the denoising processing require to start with a R^T that has a clear topology? If not, how to sample s_t? * How to select k in the k-step same-subgraph diffusion? Any general guidance? * How are the SubgDiff learning results fed into downstream tasks?

Rating

4

Confidence

4

Soundness

2

Presentation

2

Contribution

3

Limitations

None

Summary

The paper proposed SubgDiff which is a diffusion model used in self-supervised learning setup to enhance the molecular representation learning. It introduces motif enhancement during the diffusion process to force the model to learn more structure information.

Strengths

1. The idea of enhancing motif information in the diffusion process is promising. 2. The paper is well-organized and easy to follow.

Weaknesses

1. The authors directly use the baseline results from the MoleculeSDE paper in their table; however, the results for MoleculeSDE are significantly lower than those reported in the original paper. 2. The proposed method is more like a graph diffusion model than a molecular representation learning model. It is limited to representation learning within a self-supervised learning framework. It would be beneficial to explicitly state this in the abstract or introduction. 3. The paper lacks an ablation study to evaluate the contribution of each component of SubgDiff to molecular representation learning.

Questions

Please refer to Weaknesses.

Rating

5

Confidence

3

Soundness

2

Presentation

2

Contribution

3

Limitations

No

Summary

The paper presents a new denoising diffusion probabilistic model (DDPM) named SubgDiff, designed to enhance molecular representation learning by incorporating substructural information into the diffusion process. SubgDiff introduces a mask operation that selects subgraphs for diffusion, aiming to better capture the dependencies among atoms within substructures. The method includes techniques such as subgraph prediction, expectation state diffusion, and k-step same-subgraph diffusion, which together are intended to improve the learning of molecular properties related to 3D conformation. The paper claims superior performance on various downstream tasks, particularly in molecular force predictions.

Strengths

1. Utilizing subgraphs for diffusion is a novel and intriguing exploration. 2. The experiments are thorough and demonstrate the effectiveness of SubgDiff across a range of molecular prediction tasks.

Weaknesses

1. The subgraph prediction model is trained on highly specialized datasets, which might pose a risk of overfitting. 2. The model integrates multiple complex diffusion stages, which could complicate the training and debugging processes and make them difficult to optimize. Particularly, adjusting hyperparameters and verifying model stability might require additional effort. 3. Given the complexity involved in the expectation state and k-step diffusion processes, the model may have high demands on computational resources.

Questions

1. How is the stability of subgraph selection ensured during the implementation of the multi-step subgraph diffusion (k-step same-subgraph diffusion) process? Is there a risk of accumulating long-term errors due to inappropriate subgraph choices? 2. Does the training strategy for the subgraph prediction model $p_\partial(s_t|R^t)$ include handling of imbalanced data? Specifically, how does the model avoid bias towards subgraphs that appear more frequently in the training data? 3. Can this method also be applied to broader graph representation learning tasks, such as node classification and clustering? 4. Diffusing considering substructures is an interesting aspect from my perspective; besides the functional group substructures in molecular graphs, community substructures [1] are prevalent in other networks. If considering community structures during the diffusion process, what insights might the authors have? This is an interesting question, as community structures are widely present in biological and social networks. [1] 'Community detection in graph: An embedding method' in IEEE TNSE

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

This paper introduces a complex diffusion model that leverages subgraph structures to enhance molecular representation learning. However, more limitations should be discussed, such as overfitting, computational efficiency, and generalization across diverse molecular structures.

Summary

The paper proposes a diffusion-based pretraining method using subgraphs to learn enhanced molecular representations. Unlike previous methods which normally add noise to every atom, this paper proposes adding noise based on subgraphs. The method is evaluated on various downstream tasks to demonstrate its effectiveness, such as 2D and 3D property prediction tasks.

Strengths

- The paper offers an interesting perspective on existing molecular diffusion models and proposes several approaches to address them accordingly, which is inspiring. - The paper presents many experiments and analysis to illustrate the results, which could provide valuable insights to the community.

Weaknesses

- Although the proposed method effectively addresses the identified limitations, it does not provide much chemical intuition for the design. The authors claim that existing methods neglect the dependency in substructures, but the proposed method does not seem to integrate or learn the inherent molecular substructure information. The decomposition does not seem to be based on significant chemical knowledge, and the interactions or relations between various substructures are not explored. The motivation is not heavily grounded in domain knowledge. - The method is a little bit confusing. The entire training process includes many steps and three key training objectives: subgraph prediction, expectation state, and k-step same-subgraph diffusion. The paper mainly describes each component separately, but it is unclear how the three objectives are leveraged during the training. Some details are also not clear. For example, “The mask vector $s_t$ is sampled from a discrete distribution $p\_{s_t}$ (S|G)”—does $p_{s_t}$ vary for each molecule? How exactly is the distribution obtained? Also, an overall framework could help better understand the process. - The diffusion steps number is 5000, which is quite large. Molecular property prediction datasets normally contain small molecules. Are there any specific reasons for using such large steps? What are the computational costs? - The proposed method seems impractical due to its complexity, while the performance improvement is not very significant. It is difficult to justify the trade-off between model complexity and performance. Perhaps the authors could provide a more in-depth discussion to advocate for their method compared to other baselines, beyond just the prediction performance. - Some ablation studies are not conducted. For example, k seems to be an important h-param. What are the effects of different k values? Also, what about the performance of applying only one training objective?

Questions

See weaknesses.

Rating

5

Confidence

3

Soundness

2

Presentation

3

Contribution

2

Limitations

Discussed.

kind reminder to the reviewer

Dear reviewer, We kindly ask if you could inform us whether your concerns have been adequately addressed in our rebuttal, or if you have any further questions. We are committed to responding promptly to any additional inquiries you may have. Thank you for your time and valuable feedback. Best regards, The Authors

Kind reminder to the reviewer

Dear reviewer, We kindly ask if you could inform us whether your concerns have been adequately addressed in our rebuttal, or if you have any further questions. We are committed to responding promptly to any additional inquiries you may have. Thank you for your time and valuable feedback. Best regards, The Authors

Kind reminder to the reviewer

Dear reviewer, We kindly ask if you could inform us whether your concerns have been adequately addressed in our rebuttal, or if you have any further questions. We are committed to responding promptly to any additional inquiries you may have. Thank you for your time and valuable feedback. Best regards, The Authors