3M-Diffusion: Latent Multi-Modal Diffusion for Language-Guided Molecular Structure Generation

Generating molecular structures with desired properties is a critical task with broad applications in drug discovery and materials design. We propose 3M-Diffusion, a novel multi-modal molecular graph generation method, to generate diverse, ideally novel molecular structures with desired properties. 3M-Diffusion encodes molecular graphs into a graph latent space which it then aligns with the text space learned by encoder-based LLMs from textual descriptions. It then reconstructs the molecular structure and atomic attributes based on the given text descriptions using the molecule decoder. It then learns a probabilistic mapping from the text space to the latent molecular graph space using a diffusion model. The results of our extensive experiments on several datasets demonstrate that 3M-Diffusion can generate high-quality, novel and diverse molecular graphs that semantically match the textual description provided.

Paper

Similar papers

Reviewer xhrH4/10 · confidence 4/52024-05-10

Summary

This paper proposes a method for text-guided molecule generation. The method leverages a latent diffusion model to generate latent representations of molecular graphs. Further, a variational graph autoencoder is employed to decode a molecular graph from the latent representation. For text-guided generation, an LLM encoder is trained to align with the graph encoder by cross-modal contrastive learning. The output of the LLM encoder is also incorporated as a condition to control the latent diffusion model.

Rating

4

Confidence

4

Ethics flag

1

Reasons to accept

- The method is overall sound and novel. Applying diffusion models for text-guided molecule generation is an important application. - The literature review is comprehensive and effectively positions this work within the context of existing studies.

Reasons to reject

* In Introduction, the authors promised results on the GuacaMol benchmarks, yet these results are absent from the subsequent sections. * The evaluation metrics in Table 1 and Table 2 are insufficient. Why are the evaluation metrics from MolT5 (*e.g.*, BLEU, Exact, Levenshtein, MACCS) missing in this work? * **Unconditional molecule generation (Table 5)** * Since no textual condition is used, the comparison should include baselines in [1]. * Table 5 should be moved to the main body of the paper. * Table 1 and Table 2 have the same caption, yet the contents are different. The authors should specify the distinct contents of each table to avoid confusion. [1] Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models. In Frontiers in Pharmacology, 2020.

Questions to authors

cf. above.

Reviewer c27h6/10 · confidence 3/52024-05-10

Summary

The paper introduces a diffusion model that generates molecular graphs using textual descriptions of molecules as conditions. This framework first aligns the representations of molecular graphs and text descriptions. It then proceeds to train in two phases: an encoder-decoder and a multi-modal latent diffusion model.

Rating

6

Confidence

3

Ethics flag

1

Reasons to accept

- The paper proposes a novel method called 3M-Diffusion for generating diverse and novel molecular graphs from textual descriptions. The multi-modal diffusion approach that operates on an aligned latent space of graphs and text appears innovative and well-motivated to address limitations of prior work. - The experiments on multiple real-world datasets demonstrate that 3M-Diffusion outperforms state-of-the-art baselines, especially in terms of the novelty and diversity of generated molecules, while still maintaining semantic consistency with the text prompts. For example, it achieves over 146% improvement in novelty and 130% in diversity on the PCDes dataset.

Reasons to reject

- Figure 1 is confusing; the encoder and condition should be inputted together into the diffusion model. - The legends in Table 1 are incorrect for the datasets ChEBI-20 and PubChem. - Tables 1 and 2 could be merged to reduce redundancy. - How to obtain the ground truth graph of a generated molecule when calculating similarity if the descriptions of the two texts are very similar. Additional experiments, such as retrieval tasks, should be conducted to validate the alignment of representations between text and molecular graphs. - Does the case study use examples taken from the training set? - It is inappropriate to express improvements in novelty and diversity as the percentage increases. The statement '3M-Diffusion achieves 146.27% novelty and 130.04% diversity relative improvement over the SOTA method' should be rephrased to better reflect these metrics. - Due to the lack of distinctiveness in the similarity, novelty, diversity, and validity metrics, it would be beneficial to identify additional indicators for assessing the algorithm, such as plotting the distribution of logP and QED. - Why is there such a significant difference between the prompts in Figure 2 and Figure 3? The prompt in Figure 3 includes the molecular components, whereas Figure 2 only provides a simple description of properties. The disparity is too significant. The paper's statement, 'For example, the structure of ethanol can be specified using CCO, OCC, or C(O)C,' requires supporting evidence. If the method for generating SMILES is consistent, would this issue not exist?

Reviewer H3jC6/10 · confidence 4/52024-05-11

Summary

This paper studies language-guided molecule generation given text description. The proposed method consists of multiple neural networks, including a graph encoder, decoder, text encoder, and a diffusion model in the latent space. To align the representations of text and graphs in the latent space, the authors utilized contrastive loss. Experiments on conditional and unconditional molecule generation were conducted to demonstrate the effectiveness of the proposed framework.

Rating

6

Confidence

4

Ethics flag

1

Reasons to accept

1. **Less explored yet important topic.** The molecule generation using language descriptions is relatively less explored topic and it has many practical applications such as drug design and material discovery. In addition, the graph generation is relatively challenging since graphs are irregular without clear directions and they have a various number of nodes and edges. 2. **Ablation study.** The authors provide the ablation study to show the contribution of each component, including two-stage training and representation alignment.

Reasons to reject

1. **Overclaim and SOTA.** The key contribution, (iii) the proposed method outperforms SOTA methods, is questionable. First of all, the baselines did not include recent methods, and unconditional generation results are not provided in the table. 2. **No technical contribution.** The proposed pipeline is a combination of existing modules. HierVAE, Latent diffusion, GIN for graph encoder and contrastive loss for text and graph latent space alignment. Also, no significant modification/adaption of existing methods for text-guided molecule generation. 3. **Trade-off rather than improvement.** Table 1 and Table 2 show that the proposed method shows a noticeable degradation in similarity and, instead, performance gains in other metrics. This could be viewed as trade-off between metrics. In addition, Novelty, Diversity and Validity are not related to language-guidance. They are not specifically designed to evaluate the fidelity between conditions and generated samples.

Reviewer 8fJY6/10 · confidence 3/52024-05-11

Summary

This paper proposes a language model on molecule generation, well combining the existing practices in multi-modal generation: (1) train a VAE model for molecule graph generation and text descriptor generation; (2) align the representation of molecule graph and text descriptors; (3) introduce a latent diffusion model for mapping the text spaces to the latent molecular graph spaces, which then can be decoded to a molecular graph. Experiment results show the proposed method is effective, outperforming SOTA methods for both text-guided and unconditional molecule generation.

Rating

6

Confidence

3

Ethics flag

1

Reasons to accept

1. Effectiveness: The proposed method is effective and shows strong performances in both text-guided and unconditional molecule generation. 2. Well-motivated: Introducing molecular graph modeling in text-guided molecule generation is well-motivated. 3. Well writing.

Reasons to reject

1. Aligning the concept of molecular to text generation is useful, which will help us to understand the properties of unknown or new molecules. In such a case, it is more natural for me to understand the necessity of molecule-to-text generation instead of text-guided molecular generation. 2. The technical novelty is limited, but it is acceptable.

Questions to authors

1. How about using the 3D geometry for molecular modeling and introducing the protein or pocket information as a prompt for drug discovery?

Authorsrebuttal2024-05-31

Additional Experiment Results

Thanks again for your effort on reviewing our paper. Due to the word limit in the previous rebuttal part, we provide the following additional demonstrations and experiments by your suggestions: **Additional experiment and demonstrations for Q3** We present molecule-to-text retrieval (M2T) and text-to-molecule retrieval (T2M) tasks by comparing with Sci-BERT [1] and KV-PLM [2] on PubChem. Sci-BERT, pretrained on scientific texts, serves as the initial text encoder for the 3M-Diffusion model. Similarly, KV-PLM, a deep learning system, can be utilized for molecule and text retrieval tasks. Both models employ SMILES to facilitate the molecule and text tasks. Here are the following results: | Model | M2T | | T2M | | |------------|------|-------|------|-------| | | Acc | R@20 | Acc | R@20 | | Sci-BERT | 39.7 | 85.8 | 37.5 | 85.2 | | KV-PLM | 38.8 | 86.0 | 37.7 | 85.5 | | 3M-Diffusion | **61.7** | **90.8** | **63.2** | **90.2** | We observe that 3M-Diffusion can outperform Sci-BERT and KV-PLM, which verifies the effectiveness of our model for the retrieval task. [1] SciBERT: A Pretrained Language Model for Scientific Text. [2] A deep-learning system bridging molecule structure and biomedical text with comprehension comparable to human professionals.

Authorsrebuttal2024-05-31

Additional Experiment Results

Thank you once again for your comments and suggestions! Due to word limits of previous rebuttal part, we have provided additional results of more datasets for the experiments you suggested in this part. **Additional Experiments for Q1 on PCDes, Momu, ChEBI-20 and PubChem datasets:** | | | PCDes | | | | MoMu | | | | |-----------|---|------|--------|--------|---------|------|-------|--------|---------| | Metrics | | Similarity (%) | Novelty (%) | Diversity (%) | Validity (%) | Similarity (%) | Novelty (%) | Diversity (%) | Validity (%) | | Mol-instruct | | 60.86 | 35.60 | 24.57 | 79.19 | 14.89 | 97.52 | 30.17 | 68.32 | | 3M-Diffusion | | **81.57** | **63.66** | **32.39** | **100.0** | **24.62** | **98.16** | **37.65** | **100.0** | | | ChEBI-20 | | | | PubChem | | | | | |-----------|---|------|--------|--------|---------|------|-------|--------|---------| | Metrics | Similarity (%) | Novelty (%) | Diversity (%) | Validity (%) | Similarity (%) | Novelty (%) | Diversity (%) | Validity (%) | | Mol-Instruct | 65.75 | 32.01 | 26.50 | 77.91 | 23.40 | 37.37 | 27.97 | 71.10 | | 3M-Diffusion | **87.09** | **55.36** | **34.03** | **100** | **87.05** | **64.41** | **33.44** | **100** | **Additional Experiments for Q3 on all datasets (average results of all metrics are put in the following table):** | Model | PCDes | MoMu | ChEBI-20 | PubChem | |---------------|--------|--------|--------|--------| | MolT5-small | 50.06 | 51.06 | 50.06 | 44.29 | | MolT5-base | 53.96 | 46.37 | 53.97 | 46.37 | | MolT5-large | 54.77 | 54.36 | 54.36 | 54.36 | | ChemT5-small | 55.10 | 54.20 | 55.10 | 54.20 | | ChemT5-base | 55.42 | 54.56 | 55.42 | 54.56 | | 3M-Diffusion | **69.12** | **65.10** | **69.12** | **71.23** | **We can find that 3M-diffusion's average results on all metrics outperform other baselines, which show the improvement of our model.**

Reviewer H3jC2024-06-03

I appreciate the authors' detailed responses. As the experimental results, the proposed method shows strong performance. The response to the technical novelty is not convincing and novelty cannot be addressed by the response unless there exists any misunderstanding. Although this paper has limited technical contributions, this is an early attempt for molecule generation, showing great performance. I raise my rating.

Authorsrebuttal2024-06-03

Thank you for your response to our rebuttal

Dear Reviewer H3jC: Thank you very much for reviewing our paper and reading our rebuttal. We sincerely appreciate your recognition of our clarifications and the increase in your score! **Thank you also for your additional comments to facilitate further discussion, which will further improve our paper.** **In the responses below, we have carefully addressed your remaining concerns about technical novelty.** We are truly grateful for your time and your reply. Best regards, Authors

Authorsrebuttal2024-06-03

Further Responses and Clarifications to Reviewer H3jC

We sincerely thank the reviewer for the additional comments and suggestions. Please see our detailed clarifications about technical novelty below: - 3M-Diffusion studies a novel problem about text-guided molecular graph generations. **The task of generating diverse, novel molecules that match textual descriptions is challenging due to the sparsity and discrete nature of graph structures.** While there is extensive research on using variational autoencoders (VAEs) or diffusion models (DMs) for molecular graph generation to produce diverse, novel or property-related molecules, there is no existing work that leverages the capabilities of VAEs or DMs to generate molecules that are both diverse, novel, and text-matching. **We believe that the 3M-Diffusion model makes significant technical contributions and novelty by establishing a pipeline capable of generating diverse, novel and text-matching molecular graphs.** This pipeline achieves performance comparable to transformer-based models in terms of "Similarity," which measures the model's text-matching capabilities, and excels at generating diverse and novel molecules. - Furthermore, we are the first to explore the ability of diffusion models on the latent space of molecular graphs to generate novel, diverse and text-matching molecules. **Specifically, our technical contributions and novel approaches include designing the latent space and creating a suitable scoring networks for latent diffusion tailored to the latent space of molecular graphs, enabling our model to function effectively.** - Also, in our 3M-Diffusion model, **we do not directly utilize HierVAE's encoder; instead, we employ our own trained encoder that aligns graph and text representations, while only incorporating HierVAE's decoder.** Unlike HierVAE's encoder, which learns hierarchical graph representations but can’t understand text, we propose using an aligned encoder from GIN. This aligned encoder is designed to understand textual descriptions by contrastive learning. **Utilizing this encoder's representation for the decoder and making it effectively generate diverse, novel, and text-matching molecules, represents a significant technical novelty and contribution of our model.**

Authorsrebuttal2024-06-03

A sincere and kind reminder to the reviewer

Dear Reviewer xhrH: We gratefully appreciate your time in reviewing our paper and your comments. We have made extensive efforts to address your comments and believe that they adequately address all your concerns. The reviewer's comments are mainly about some clarifications and are not fatal to the contributions of our paper; we believe that the reviewer's insightful comments can be easily and effectively addressed in the final version. With only four days left in the discussion period, we would like to confirm whether there are any other clarifications they would like. If the reviewer's concerns are clarified, we would be grateful if the reviewer could increase the score. Many thanks for your time; we are extremely grateful. The authors of "3M-Diffusion: Latent Multi-Modal Diffusion for Language-Guided Molecule Generation"

Authorsrebuttal2024-06-05

Kind reminder to reviewer xhrH: the author-reviewer discussion period is coming to an end.

Dear Reviewer xhrH, We want to sincerely thank you again for your time and comments! We have made extensive efforts to address your comments and carefully considered your suggestions point-by-point and added unconditional molecule generation you request in the rebuttal. We would like to kindly remind you that we are approaching the end of the author-reviewer discussion. In light of our rebuttal, we kindly ask if you could consider increasing your score. Thank you very much once again, and we look forward to hearing back from you if you have further comments. Thank you very much for your time. Best regards, The Authors

Authorsrebuttal2024-06-05

General Response by Authors

Dear Reviewers, We sincerely thank all the reviewers for their thoughtful feedback and their time. We are encouraged to see all reviewers agree that our paper is a novel, interesting, and effective work with strong experimental results. In our work, we propose the first multimodal diffusion approach to generating molecular graphs from their textual descriptions, which is critical for molecule generation or drug discovery with textual guidance. Our proposed 3M-Diffusion effectively aligns the latent spaces of molecular graphs and textual descriptions to offer a text-molecule aligned latent diffusion model to generate high-quality, diverse, and novel molecular graphs that match the textual description provided. We demonstrate that our 3M-Diffusion has great empirical performance on both text-guided molecule generation and unconditional molecule generation compared with baselines. These empirical results and a novel perspective for text-guided molecule generation were found to be particularly interesting by all reviewers. We have made our greatest efforts to incorporate all questions, suggestions, and comments in our response to each reviewer, and have prepared a point-by-point response for each. Briefly, we have addressed key themes in the reviews, including additional comparisons with baselines, retrieval tasks, misunderstandings, and minor questions. Below, we summarize the major responses, while we address the comments of each reviewer separately. - We added the additional comparisons with baselines for text-guided molecule generation and unconditional generation (Reviewer H3jC, xhrH). - We added retrieval tasks for our text and graph encoder (Reviewer c27h). - We added additional clarification about technical novelty and contributions (Reviewer H3jC, 8fJY). - We compared 3M-Diffusion with the baseline by plotting the distribution of logP and QED. (Reviewer c27h). For the other questions, we have provided individual responses to each one. We will update the paper to include these additional experiments and clarifications in response to your comments. We extend our sincere thanks to all the reviewers once again for their time and efforts. Best regards, Authors

Program Chairsdecision2024-07-10

Decision

Accept

© 2026 NYSGPT2525 LLC