How Molecules Impact Cells: Unlocking Contrastive PhenoMolecular Retrieval

Summary

The paper introduces MolPhenix for tackling the contrastive phenomolecular retrieval problem. MolPhenix employs a uni-modal pretrained phenomics model, an inter-sample similarity-aware loss function, and conditions on the representation of molecular concentration. This approach effectively addresses challenges such as experimental batch effects, inactive molecule perturbations, and encoding perturbation concentration. Experimental results demonstrate the model's effectiveness in the retrieval task.

Strengths

- The paper is well-written and the motivation is clear. - The identified challenges are highly relevant and significant in the biological context, and the proposed methods are logical and well-conceived. - The experimental results indicate substantial improvements over baselines across various settings in the retrieval task.

Weaknesses

- In Table 2 and Table 4, there is a significant performance increase for DCL, CWCL, SigLip, and S2L compared to other baselines. The source of these improvements is unclear. Conducting an ablation study for each of the three components of the method, corresponding to the three challenges, would provide more insights. - The paper primarily compares MolPhenix with CLOOME and a few other general domain objectives. However, numerous related studies specifically within the molecular-phenotype contrastive learning domain [1-6] are not discussed or compared. - The paper emphasizes the retrieval task, with experimental results showing only the top 1% recall accuracy. This limits the overall impact of the findings. Considering broader application scenarios based on the proposed method could enhance the paper's significance. [1] Cross-modal graph contrastive learning with cellular images. bioRxiv 2022. [2] Contrastive learning of image- and structure-based representations in drug discovery. ICLR MLDD 2022. [3] Molecule-Morphology Contrastive Pretraining for Transferable Molecular Representation. ICML CompBio 2023. [4] MMCL-CDR: enhancing cancer drug response prediction with multi-omics and morphology images contrastive representation learning. Bioinformatics 2023. [5] Removing Biases from Molecular Representations via Information Maximization. ICLR 2024. [6] Learning Molecular Representation in a Cell. arXiv 2024.

Questions

- The retrieval accuracy results in the paper are reported as top 1% accuracy. However, this metric may be less informative when the retrieval set is very large, as the top 1% can still include a substantial number of molecules. Could the authors also provide results for top N accuracy, where N = 1, 10, and 100?

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

The authors have noted several limitations in their methods and experiments. Additional limitations are listed in the **Weaknesses** and **Questions** sections.

Summary

The paper makes a significant step forward in the task of phenotype-molecular retrieval–the task to find the molecule applied to perturb a set of cells given a microscopy readout. This problem can be modeled using multi-modal contrastive learning. The proposed MolPhenix method leverages pre-trained foundation models to encode the microscopy images and molecular structures. Then a novel contrastive loss is used which takes into account domain specific issues like batch effect and different concentrations of the applied molecules. Through thorough evaluations and ablations the authors show increased performance over a baseline.

Strengths

- The paper tackles a task relevant for drug discovery and shows a significant step forward in predictive performance - The paper is very well written - The authors perform extensive experiments on held out molecules, phenotypes and dataset - The CLOOME baseline is tuned convincingly - The proposed guidelines for the task are supported by ablation studies - The paper combines and improves upon SOTA methods in multi-modal contrastive learning - The authors show domain knowledge by incorporating reasonable data processing and training strategies (batch effect removal + undersampling of inactive molecules)

Weaknesses

- The authors focus on retrieval instead of evaluation of the latent space. Given the expressiveness of the leveraged foundation models, generating phenotypes or molecules should be possible. It would be great if the authors could comment on this or other explorations of the phenotypic space. - The authors train on a private dataset which makes the paper almost not reproducible. However, due to detailed model descriptions, the guidelines could be evaluated on other datasets.

Questions

- Please elaborate on generative or mechanistic opportunities of the model - You argue MolPhenix cannot be applied to images. Please elaborate since instead of ph-1 a simple non domain-specific image feature extractor could be used. Have you studied the results? - Please comment on which model weights of ph-1, Mol-1, and MolPhenix are publicly available or you will be releasing

Rating

8

Confidence

3

Soundness

4

Presentation

4

Contribution

3

Limitations

- Limited reproducibility due to private datasets and the model weights likely not being released

Summary

The paper introduces MolPhenix, a framework for contrastive phenomolecular retrieval that integrates phenomic data and molecular structures into a joint embedding space. Key contributions include combining phenomic and molecular data for improved retrieval accuracy, proposing effective training guidelines, and addressing cumulative concentrations and label noise. The framework demonstrates significant performance improvements over existing methods, particularly in zero-shot settings, and is supported by comprehensive experiments and ablation studies.

Strengths

The paper is well-structured and comprehensive, showcasing rigorous experimentation through extensive ablation studies, comparisons with baseline methods, and evaluations. Its strength lies in the detailed and methodical approach to validating the MolPhenix framework, providing strong evidence for its effectiveness and potential applications. The experimental design is robust, demonstrating the framework's superiority in various scenarios and contributing valuable insights to the field. Additionally, the integration of molecular and phenomic data into a joint multi-modal embedding offers a fresh perspective, enhancing the overall impact and originality of the work.

Weaknesses

1. The clarity of this work could be significantly improved. The task and background are not well stated, and the related work is not adequately introduced. Many biological terminologies are mentioned, creating a large gap between the introduction and the real dataset. 2. The paper could benefit from including comparisons with a broader range of state-of-the-art methods, particularly those from adjacent fields such as multi-modal representation learning in genomics and proteomics. This would highlight the specific innovations of MolPhenix more distinctly. 3. The significance of the work could be more explicitly articulated and demonstrated. Although terms like drug discovery are mentioned, the real-world impact is not clear for readers.

Questions

1. For molecular representations, did you consider combining GNNs and fingerprints? In other tasks, such as property prediction, their combination often surpasses the performance of each method individually. 2. Did you consider to report and compare metrics for top-5 and top-10 recall?

Rating

5

Confidence

3

Soundness

3

Presentation

2

Contribution

3

Limitations

The authors have adequately addressed the limitations.

Summary

This paper introduces MolPhenix, a model designed to learn a joint latent space between molecular structures and microscopy phenomic experiments, addressing the challenge of contrastive phenomolecular retrieval. The authors point out three key challenges in this domain: limited paired data & batch effects, inactive molecular perturbations, and variable concentrations. To address these issues, they propose a set of guidelines: 1) leveraging a pre-trained phenomics foundation model (ph-1), 2) mitigating the impact of inactive molecules through undersampling and a novel soft-weighted sigmoid locked loss (S2L), and 3) encoding molecular concentration both implicitly (within the S2L loss) and explicitly (by parsing dosage concentration). The primary experiments focus on the task of phenomolecular retrieval. For active molecules, MolPhenix achieves up to 77.33% top-1% retrieval accuracy, representing an 8.1-fold improvement over the baseline. The paper includes necessary ablation studies and evaluations across multiple datasets, demonstrating the effectiveness of their approach in both cumulative and held-out concentration settings.

Strengths

1. Novel approach: The paper introduces MolPhenix, a comprehensive framework that addresses several key challenges in contrastive phenomolecular retrieval by combining multiple innovative techniques. This new framework demonstrates notably strong performance. 2. Concentration encoding: The study explores both implicit and explicit methods for encoding molecular concentration, enhancing the model's ability to capture dose-dependent effects and generalize across concentrations. The explicit concentration module and inactive molecule undersampling techniques may inspire future multi-molecule research. 3. Detailed ablation: The paper provides an in-depth analysis of results, including the impact of various components (e.g., loss functions, concentration encoding methods) on model performance, presented in both the main body and appendix.

Weaknesses

The following are just some minor concerns, listed in descending order of importance: 1. Insufficient justification for S2L loss: While the paper introduces the S2L loss as a key contribution, it lacks a thorough theoretical analysis explaining why this loss function is suitable for the phenomolecular retrieval task. A more rigorous mathematical analysis of S2L would strengthen the paper's contribution. 2. Inadequate ablation of pretrained components: As a phenomolecular retrieval "framework", the study should include ablation studies using different molecular and phenomic encoders. However, it appears that experiments were conducted only with fixed molecular GNN and phenomic models. 3. Oversimplified treatment of batch effects: The paper claims to address batch effects through embedding averaging, but this approach may be too simplistic. A more detailed explanation of why simple averaging can alleviate batch effects, or some minor adjustments to this batch effect removal procedure, would be beneficial.

Questions

Regarding the equality of involved datasets: 1. How would the model's performance vary if only the open-source RxRx3 data or only the private novel data were used? 2. Given that many components of MolPhenix are publicly pretrained models, is it feasible to construct a comparable model using solely open-source resources?

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors have properly adressed the limitations in their paper.

Summary

This work focuses on predicting the molecular impact on cellular functions and investigates the problem of contrastive phenomolecular retrieval. It introduces MolPhenix, a model that leverages a joint latent space between molecular structures and microscopy-based phenomic experiments using contrastive learning. The main contributions include the use of a pre-trained phenomics model, a novel inter-sample similarity-aware loss (S2L), and molecular concentration conditioning, leading to a significant improvement over the previous state-of-the-art in zero-shot molecular retrieval of active molecules.

Strengths

The introduction of the MolPhenix model, which utilizes a pre-trained phenomics model, a novel inter-sample similarity-aware loss (S2L), and molecular concentration conditioning, is original and demonstrates improvements over existing methodologies. The experimentation of this work is comprehensive and detailed to justify the proposed method. The paper is well-structured and clearly written with clear explanations of the methodologies. The problem investigated is important in the field of drug discovery and has a growing interest.

Weaknesses

For the captions of Table 2~5, it is better to mention the experimental setting (cumulative concentration & held-out concentrations). The current captions are nearly the same (Table 2 vs. Table 4, and Table 3 vs. Table 5), which is kind of confusing.

Questions

For the pretrained GNN, are there any specific advantages to choosing the current one, given that there are many other pretrained GNNs that can be used to extract molecular representations? Are there any discussions regarding the results of different concentration encoding choices? In a cumulative concentration setting, one-hot performs the best, while in a held-out concentration setting, not using any explicit concentration is the best choice overall.

Rating

7

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors have adequately addressed the limitations.

Reply to rebuttal

I want to thank the authors for their detailed responses to my questions.