Molecule Generation with Fragment Retrieval Augmentation

Summary

The paper introduces a novel fragment-based molecule generation framework called Fragment Retrieval-Augmented Generation (f-RAG). This framework aims to address the limitations of existing fragment-based molecule generation methods, which often struggle to explore beyond the existing fragments in their databases. f-RAG utilizes a pre-trained molecular generative model to propose additional fragments from input fragments, completing and generating a new molecule. It retrieves two types of fragments: hard fragments, which are explicitly included in the newly generated molecule, and soft fragments, which serve as references to guide the generation of new fragments through a trainable fragment injection module. To further extrapolate beyond the existing fragments, f-RAG updates the fragment vocabulary with generated fragments through an iterative refinement process. This process is enhanced with post-hoc genetic fragment modification, allowing f-RAG to maintain a pool of fragments and expand it with novel and high-quality fragments through a strong generative prior. This approach enables f-RAG to achieve an improved exploration-exploitation trade-off in fragment-based molecule generation.

Strengths

1. f-RAG uses a pre-trained molecular generative model to propose new fragments, allowing new fragments to be generated 2. The approach retrieves both hard fragments that are directly incorporated into the new molecule and soft fragments that guide the generation process, enhancing the diversity and effectiveness of the generated molecules. 3. f-RAG updates the fragment vocabulary with generated fragments through an iterative process, continuously improving the quality and novelty of the fragments used for molecule generation.

Weaknesses

1. The improvement of f-RAG compared to Genetic GFN is not considered large enough. In many tasks, Genetic GFN actually performs much better than f-RAG, which raises the concern of generalization of the proposed method. 2. Some notations are not clear enough. For example, in Table 2, arrows could be used to illustrate lower-better or higher-better.

Questions

1. Could the authors explain the potential reason that f-RAG falls behind genetic GFN?

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Yes, the limitation is well-discussed.

Summary

The paper proposed a fragment retrieval-augmented generation for molecule discovery, namely f-RAG. f-RAG retrieves two types of fragments, i.e., hard fragments and soft fragments. Hard fragments serve as build blocks that are explicitly included in the newly generated molecules, while soft fragments guide the generation of new fragments through a trainable module.

Strengths

The work is well written. The experiments show the effectiveness of the proposed method.

Weaknesses

1. The novelty of the work is limited. Very similar works have been published, such as [1], [2], and [3]. They basically follow the same pipeline, which generates new molecules through LLMs, GAs, or both. Compared to these works, this work does not seem to bring new insights. 2. Moreover, the so-called soft and hard fragments are not new either. In reference [1], retrieved exemplar molecules are used as inputs to guide the generation of new molecules through trainable networks. Therefore, the concept of soft fragments is not novel. 3. In addition, reference [2] also used GAs and LLMs to generate molecules. The author needs to further clarify the differences between the two works. 4. The baselines used by the authors are not the latest. The authors should consider incorporating LLMs related molecular generation methods into the comparison. 5. Why did the author use SAFE-GPT instead of other chemical language models? Can this method be extended to other chemical large language models? 6. Some key details in genetic algorithms are missing, such as how to use mutation and crossover to generate new molecules. [1] Wang, Zichao, et al. "Retrieval-based controllable molecule generation.", ICLR, 2023. [2] Lee, Seul, et al. "Drug Discovery with Dynamic Goal-aware Fragments." Forty-first International Conference on Machine Learning, 2023 [3] Wang, Haorui, et al. "Efficient Evolutionary Search Over Chemical Space with Large Language Models." arXiv preprint arXiv:2406.16976 (2024).

Questions

See weakness.

Rating

3

Confidence

5

Soundness

2

Presentation

2

Contribution

1

Limitations

No potential negative societal impact

Summary

The paper introduces f-RAG, a novel framework for fragment-based molecular generation that integrates hard and soft fragment retrieval and genetic fragment modification. It aims to improve the exploration-exploitation trade-off in drug discovery by leveraging existing molecular fragments and exploring beyond the existing chemical space. On each generation, two hard fragments are sampled which will be ensured to appear in the generated molecule. Then several soft fragments are sampled to derive an embedding via a pretrained chemical language model as the guidance for generation. Another round of genetic algorithm is also implemented to further explore the neighborhood of the generated high-scoring molecules. The authors have conducted extensive experiments on various molecular optimization tasks, demonstrating f-RAG's effectiveness in generating molecules with improved optimization performance, diversity, novelty, and synthesizability.

Strengths

1. The paper tries to tackle an important problem of fragment-based molecular generation, namely exploring further chemical spaces beyond known fragments. During generations, novel and high-scoring fragments will be dynamically updated into the vocabulary so that this kind of "novelty" will be passed on to further generations, enlarging the explorable chemical space. 2. The retrieval-augmented generation provide explainability to some extent, therefore has some ensurance on the quality of the generated molecules.

Weaknesses

1. Integrating both language models and genetic algorithms into generation might result in huge computational cost, which will affect the practical applications of generative models. The authors should discuss and analyze the computational efficiency as well as its trade-off with the performance. 2. Commonly the RAG process will retrieve related information from large-scale database (vocabulary) through some techniques (e.g. Vectorize the data and implement query-key matching). However, in the proposed method, both the hard and soft fragments are randomly sampled from the vocabulary, which leaves great burdens on the construction of high-quality and relevant databases for each task or property. 3. The implementation details are missing, especially for the baselines which incorporate genetic algorithms, whose performance is closely related to the values of parameters like population size and number of cycles.

Questions

1. Why can all the molecules be decomposed into an arm-linker-arm form? Does this formalization impose additional biases on the chemical space which can be explored? For example, what will happen if I want to generate benzene with small substituents (e.g. toluene).

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

The property-related fragment vocabulary can only be built if the property score can be decomposed into fragment-level sums, which limits the applications of the proposed method.

Summary

This study proposes a fragment retrieval-augmented generation framework for molecular designs based on language models. The arm and linker vocabulary are constructed by fragments that have top average contribution to the given property. The hard fragments and a pool of soft fragments are retrieved from the vocabulary whose embeddings are fused and used for the generation. The generated molecules and the vocabulary are iteratively refined and augmented to further enhance the performance. The authors report competitive performance on multiple properties and generative tasks, including both single and multiple objectives.

Strengths

This work formulates a novel framework for molecule designs and serves as a useful platform for future explorations. Though most of the techniques used in this study are not novel, the authors demonstrate effective ways of combining them to improve the performance. Specifically, the use of soft fragment pools and the genetic refinement make a good balance between exploration and exploitation. The authors provide comprehensive evaluation, comparison and ablation results, and also show competitive performance in multi-objective optimization, which is more relevant to real-world drug discovery applications.

Weaknesses

The work is overall solid and I don't have major concerns except some minor questions.

Questions

1\. As the vocabularies are defined on the target property, is the fragment injection module also trained for each property? Would a universal model also work for the scenario? 2\. Line 242: the multi-objective optimization is performed using a unified score which is the product of all objectives. This may cause some problems such as high scores in one objective overshadowing others. How would this compare with, for example, selecting fragments that have higher scores for all three objectives? 3\. Table 3: as shown in Table 1, Genetic GFN does rather well in other tasks, so it would be better to also compare with it here. 4\. Line 275 and Table 8: the no-GA setting has substantially higher diversity than all other settings. This is somewhat counterintuitive as GA expands the fragment vocabulary and could potentially lead to more diverse and novel molecules. Thus, the no-GA setting should have lower diversity just like it has lower novelty. What could be the cause of this observation? 5\. Fig 5c: what are the actual values of the results?

Rating

7

Confidence

4

Soundness

4

Presentation

4

Contribution

3

Limitations

The authors have properly addressed the limitations.

Summary

Fragment-based drug discovery methods are limited in their exploration beyond existing database fragments, as they primarily reassemble or slightly modify the given fragments. This paper introduces a new approach, fragment retrieval-augmented generation (f-RAG), which retrieves two types of fragments—hard fragments and soft fragments—from a fragment vocabulary to achieve an improved exploration-exploitation to address this limitation.

Strengths

This paper introduces a novel molecular generative framework that combines fragment-based drug discovery (FBDD) and retrieval-augmented generation (RAG). This paper proposes a retrieval augmentation strategy that operates at the fragment level, utilizing two types of fragments to provide fine-grained guidance. This approach aims to achieve a better exploration-exploitation trade-off and generate high-quality drug candidates.

Weaknesses

1. This paper claims the f-RAG approach improve the exploration-exploitation trade-off. However, there is no experiment demonstrate this point. 2. No limitation is discussed. 3. Novelty and contribution is a concern, from the RAG part, it seems the main difference compared to Want et al. [42] is f-RAG dealing with fragment instead of molecule. 4. In addition, the critical part, SAFE-GPT[34], is a previous work. 5. f-RAG is built on a pre-trained backbone molecular language model, and it relies heavily on the generation performance of this backbone. This also means that the method delegates the challenging task of molecule generation to a large model.

Questions

see weakness.

Rating

4

Confidence

3

Soundness

2

Presentation

3

Contribution

2

Limitations

f-RAG is built on a pre-trained backbone molecular language model, and it relies heavily on the generation performance of this backbone. This also means that the method delegates the challenging task of molecule generation to a large model.

Thanks for the response

Thank the authors for the detailed response, which has alleviated some of my concerns. I would like to maintain my recommendation of weak acceptance, and hope the authors good luck in addressing the concerns of the other reviewers.