Aligning Target-Aware Molecule Diffusion Models with Exact Energy Optimization

Summary

This paper focuses on generating ligands with desired properties, such as high binding affinity, for the protein-conditioned ligand generation task. The authors propose a Preference Optimization (PO)-based fine-tuning method for pre-trained generative models. They extend DPO (Data-driven Preference Optimization) to propose exact energy optimization that directly utilizes reward values in the loss function. Experiments demonstrate that the proposed method can generate ligand molecules with the desired properties.

Strengths

* Comprehensive ablation studies address readers' concerns. * An efficient objective function is designed for tasks where reward values are directly available, such as binding energy. * The proposed method's effectiveness is validated using multiple pre-trained models, both IPDiff and TargetDiff, demonstrating its generalizability. * The investigation into preference pair selection is interesting and shows room for future research. * The proposed method is well-designed. * The paper is well-structured and easy to read.

Weaknesses

* The practical impact of the achieved improvement needs to be clearly quantified. * There is no consideration or results for comparison or combination with methods other than DPO, such as IPO or SLiC.

Questions

* The proposed objective function seems to be heavily influenced by the scale of the reward value r. Was any adjustment necessary? In particular, was simple addition sufficient for the Affinity + SA experiment that combined multiple rewards? Additionally, is it easy to incorporate other rewards, such as diversity? * The paper states that if $r^w \gg r^l$, then $\sigma(r^w - r^l) \approx 1$. However, wouldn't it be more accurate to say that if $r^w - r^l \ge 1$, then $\sigma(r^w - r^l) \approx 1$? What is the actual distribution of $r^w - r^l$ in the training data?

Rating

7

Confidence

4

Soundness

3

Presentation

4

Contribution

3

Limitations

The proposed objective function cannot be used when reward values are not explicitly available, making it unsuitable for cases involving manually binary-labeled results.

Summary

This paper presents ALIDIFF, a framework that aligns pre-trained target-aware molecule diffusion models with desired functional properties using preference optimization. The key contribution of ALIDIFF is the Exact Energy Preference Optimization method, which precisely aligns diffusion models to regions with lower binding energy and better structural rationality based on user-defined reward functions. Extensive experiments on the CrossDocked2020 benchmark showcase ALIDIFF's strong performance, generating molecules with state-of-the-art binding energies and maintaining competitive molecular properties. By incorporating user-defined reward functions and improving the Exact Energy Preference Optimization method, ALIDIFF achieves significant advancements in binding affinity.

Strengths

* ALIDIFF utilizes a preference optimization approach to effectively steer the model towards generating chemically and structurally relevant molecules. * It Demonstrates the effectiveness and robustness of the method through extensive experiments, showing superior binding energies while maintaining strong molecular properties. * The paper employs AutoDock Vina for accurate binding energy evaluations, ensuring practical outcomes in drug discovery applications.

Weaknesses

* The method leverages IPDiff and further optimizes it using RL, which makes performance improvements easier. Consequently, the methodological novelty appears marginal. * The paper lacks specific information on the time required for pre-training, fine-tuning, and binding energy computations via AutoDock Vina. This absence makes it difficult to fully assess the advantages and limitations of the proposed method. - AutoDock Vina is known to be time-consuming, and RL also requires significant computational time. Including these time details would help illustrate both the strengths and constraints of the proposed approach.

Questions

* Train/test split is 65K/100. It seems that the test set is too small compared to the train set. Is there any reason for that? * How much time does it take to pre-train, fine-tune and binding energy computations via AutoDock Vina?

Rating

5

Confidence

2

Soundness

3

Presentation

2

Contribution

3

Limitations

Yes

Summary

This paper proposes a novel alignment framework, known as ALIDIFF, to align pretrained target diffusion model with preferred functional properties for structure-based drug design. ALIDIFF shifts the target-conditioned chemical distribution towards regions with higher binding affinity and structural rationality, specified by user-defined reward functions, via the preference optimization approach. The experiments show that this method can generate ligands with better binding energy and maintain competitive molecular properties.

Strengths

- This method is novel in using energy preference optimization framework to align molecule generative models with desirable properties in order to generating molecules with high binding affinity to binding targets. - They analyze the overfitting issue in the preference optimization objective, and propose an improved exact energy optimization method to yield an exact alignment towards target distribution shifted by reward functions.

Weaknesses

- Lipinski metric should also be reported in Table 1. This is a commonly used metric for measure drug-likeness in previous work. - Fragment-based methods, such as FLAG and DrugGPS, is missing in Table 1. For comprehensive comparison, I recommend add those two baselines. - Generated molecules have inferior drug-likeness properties. Isn't it possible to also align the pre-trained diffusion model with high drug-likeness region?

Questions

Please refer to the weakness part.

Rating

4

Confidence

3

Soundness

2

Presentation

2

Contribution

3

Limitations

Limitation is discussed.

Thanks for the clarification

Thank you for the detailed rebuttal. I have read it thoroughly. Based on the clarified points, I believe that while the methodological novelty may be limited, the practical value is significant. Therefore, I will raise my score to 5. Thank you for the excellent research.

Thanks

Thank you for the detailed response. It has addressed my concerns. I still believe that this work is worth presenting. > Mathematically, for the sigmoid function, ... Sorry, it is my misunderstanding. You are correct.

Looking forward to feedback during the reviewer-author discussion period

Dear reviewer 5bxb, We would like to first express our sincere gratitude for your time and effort in reviewing our paper. We appreciate your valuable suggestions and will incorporate the discussions and results to the final version. Considering the author-reviewer discussion is ending very soon, could you kindly check our responses and let us know if you have further concerns? We are more than willing to address any other concerns or questions. We would greatly appreciate it if the reviewer would consider adjusting the score, on the basis of our response and other review comments.   Thanks again for your constructive reviews!   Sincerely, Authors

Looking forward to feedback during the reviewer-author discussion period

Thanks again for your time spent reviewing our work and the constructive comments and suggestions. We provided evaluations of two related works and our results showcase that the proposed method consistently outpeforms all baselines on binding affinity metrics, which have also been acknowledged by other reviewers. We also added lipsinki as another measure of drug-likeness and conducted new experiments on aligning molecules with drug-likeness propoerties, as suggested. As the discussion period draws to a close, we would be grateful for your further feedback on our clarifications and recognition of our research's contributions. Thank you again for your valuable suggestions! Best, Authors