MuSe-GNN: Learning Unified Gene Representation From Multimodal Biological Graph Data

Summary

This paper introduces MuSe-GNN, a model for learning gene embeddings from single-cell sequencing and spatial transcriptomic data that is based on multimodal machine learning and deep graph neural networks. While incorporating regularization with weighted similarity learning and contrastive learning to learn cross-data gene-gene relationships, the proposed method creates informative graph structures for model training and the generation of gene representations, which can make the learned gene representations contain functional similarity across various contexts in a joint space. The results of tests on diverse multimodal biological datasets show that MuSe-GNN can efficiently learn the functional similarity of genes across tissues and methods and surpasses current gene embedding learning models across several metrics. This model can aid in the investigation of the pathogenic processes underlying conditions like COVID and lung cancer.

Strengths

1. The proposed model learns the gene expression, which incorporates the regularization with weighted similarity learning and contrastive learning to learn cross-data gene-gene relationships. The proposed model can analyze the large-scale multimodal biological datasets. 2. The model is applied to analyze COVID and cancer datasets, which helps to unveil potential disease resistance mechanisms or complications based on specific differentially co-expressed genes. 3. MuSe-GNN has obtained significant improvement from 24.2% to 100.4% in comprehensive assessment. 4. The overall pipeline and the mechanisms are presented in detail.

Weaknesses

1. What is the difference between this paper and the Geneformer [1]. What about the comparisons of the performance of the experiments if they can be compared together? 2. In Figure 1, why Gene2vec and GIANT failed to learn the gene similarity? The embeddings are clustered by datasets. [1] Theodoris, Christina V., Ling Xiao, Anant Chopra, Mark D. Chaffin, Zeina R. Al Sayed, Matthew C. Hill, Helene Mantineo et al. "Transfer learning enables predictions in network biology." Nature (2023): 1-9.

Questions

See above.

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

Yes, the authors have discussed their limitations and potential impact of gene embeddings for diseases, COVID.

Summary

The authors proposed a novel deep graph neural network, named MuSe-GNN, to learn gene representations from single-cell sequencing and spatial transcriptomic data. The idea is to construct gene graphs from single-cell sequencing and spatial transcriptomic data, and then learn gene embeddings via cross-graph transformer.

Strengths

The tasks of learning gene embeddings from multi-context sequencing profiles is of interest especially in the bioinformatic domain. The empirical results seem to indicate that the approach can be applied to different analysis frameworks and the gene representations learned by their approach are highly versatile. Writing and presentation skill is well.

Weaknesses

This idea is not new, and the models they used are all well established. Lack of comparison with related works. Lack of ablation studies.

Questions

One additional ablation experiment would be of interest: how much would the performance be impacted if you induce the embedding using only part of the modules. They can compare their method with other deep learning-based multimodal representation learning models, such as [1]. [1] Lin X, Tian T, Wei Z, et al. Clustering of single-cell multi-omics data with a multimodal deep learning method[J]. Nature communications, 2022, 13(1): 7705.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

N/A.

Summary

This paper addresses the heterogeneity problem in the gene representation learning from multi-context biomedical sequencing profiles. Specifically, gene connections are formulated through co-expression network, then the proposed MuSe-GNN utilizes cross-graph Transformer to generate gene embeddings, while designing a contrastive-learning based regularization term to integrate multi-modal data.

Strengths

1. The paper is well-motivated. Effective gene representation learning from multi-context profile datasets is critical real-world biomedical applications. 2. The experimental results demonstrate that MuSe-GNN achieves the state-of-art performance and brought a significant improvement. 3. Bioinformatical analysis on identifying significant pathways, diseases, and causal networks provides biologically-informed insights .

Weaknesses

1. The novelty of the proposed method is limited. 2. The methodology part is not adequately described. The cross-graph Transformer section is hard to follow and Figure 2 can be improved.

Questions

Minors: The application area is rather specialized, so this paper may be of interest to a smaller subset of the NeurIPS community.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

Yes.

Summary

The paper describes an approach that combines Multimodal Machine Learning and Deep Graph Neural Networks to learn gene representations from multi-omics and multi-tissue data. The main issue that this paper tries to address is that existing approaches fail to obtain gene representations that are consistent across modalities (esp. from different data sources but same tissue). To accomplish this goal, the authors propose a model called MuSE-GNN that takes as input graph co-expression networks extracted from different modalities and projects them into a unified and consistent latent space to extract gene embeddings. These gene embeddings are then used for several downstream tasks such as pathway analysis, causal network analysis, and disease analysis. In particular, the model is learned through a loss function composed of three terms: a co-expression graph reconstruction term, a weighted similarity term (where the pairwise similarity of gene representations across modalities is maximized), and a self-supervised contrastive term (to cluster together functionally similar genes and pull apart functionally different genes). The model is evaluated on a benchmark consisting of different tissues against 9 other methods, where it shows excellent performance on different metrics. In a subsequent experiment, the authors show that MuSE-GNN embeddings successfully relate to functional groups in human data. Lastly, they present a case study on COVID-19 where they run the model on pancreatic human cells of healty vs diseased COVID patients to identify differentially expressed genes, showing (with gene enrichment) that these genes are related to immune system activity and that some of them are part of known causal networks of the regulatory activity of the immune system.

Strengths

This paper is well-written and easy to follow. It studies a relevant problem (how to obtain consistent gene embeddings across modalities) and proposes a quite elegant solution (make them consistent mainly through a tailored loss function). The experimental results are good and the ablations (in the appendix) are exhaustive and very informative.

Weaknesses

It seems that the competitors have been evaluated with default hyper-parameters, while the hyper-parameters of MuSE-GNN have been tuned. This is unfair, since the results of MuSE-GNN are optimized to the experimental settings of this paper, while those of the competitors are not. This might be addressed by tuning (some of) the hyper-parameters of the competitors, although I understand it might be difficult given the time constraints of the review process.

Questions

- from the code, I can see that the competitors are trained with random seeds in the range [0, 10) while I can't see the same for MuSE-GNN. Can you confirm that MuSE-GNN has been trained with the same random seeds and thus the results are comparable? - can you explain better this sentence in the appendix: "Methods such as WSMAE, GAE, VGAE, and MAE, which are based on MLP/GNNs and inspired by other methods, are configured with the best hyperparameters based on parameter searching to ensure fairness."? - Your model uses 349M parameters. How many parameters GIANT uses? If they don't match, can you train a version of MuSE-GNN with a comparable number of parameters to GIANT (or alternatively, a version of GIANT with a number of parameters comparable to MuSE-GNN)?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

3 good

Limitations

Not discussed. I don't see many limitations of this approach besides getting data of proper quality (which is independent of the contribution). Perhaps, as mentioned by the authors, the model is quite "heavy" in terms of number of parameter and one needs to bring down the embedding dimension to avoid OOM errors. No negative societal impact of concern.

Thanks

I have read your rebuttal and appreciated your effort. I now think that this paper is mature enough to make it to the Neurips bar and have changed my score accordingly. Good luck!