PhyloGen: Language Model-Enhanced Phylogenetic Inference via Graph Structure Generation

Summary

The authors introduce PhyloGen, a new method that uses a pre-trained genomic language model to generate phylogenetic trees without relying on evolutionary models or aligned sequences. PhyloGen treats phylogenetic inference as a conditionally constrained tree structure generation problem, jointly optimizing tree topology and branch lengths through three modules: Feature Extraction, PhyloTree Construction, and PhyloTree Structure Modeling. A Scoring Function guides the model towards stable gradient descent. The method achieves state-of-the-art performance when evaluated on benchmark datasets.

Strengths

1. The framework is novel in that it leverages a pre-trained genomic language model instead of relying on aligned sequences. 2. The method achieves state-of-the-art performance evaluated on real-world datasets.

Weaknesses

1. Ablation Experiments Clarity The ablation experiments are unclear. My main concern is the lack of clarity regarding why the proposed method significantly outperforms other methods. Specifically: (1) Table 5: This table shows the average metrics across the eight datasets. However, Table 1 and Table 2 only shows the metrics for each dataset individually without any average metrics. It’s unclear what the performance gap is between the ablation model and the baseline methods when KL or S are removed. (2) Figure 6: The ablation studies in this figure only show the metrics for the DS1 dataset. Please include the average metrics across all eight datasets. To improve the clarity of the ablation experiments, I strongly suggest making the evaluation settings consistent in Table 1, Table 2,Table 5, and Figure 6. At the very least, include the average metrics across the eight datasets. Additionally, provide more explanations based on the complete ablation experiments to clarify why the proposed method significantly outperforms other methods. 2. Claim on Aligned Sequences vs. Unaligned Sequences The claim that unaligned sequences better reflect actual sequence variation than aligned sequences is questionable. Generally, aligned sequences with direct site-to-site comparisons are more effective in comparative sequence analysis. The authors need to provide more direct experimental evidence to support the advantage of using unaligned sequences. 3. Writing Issues There are many writing issues. For instance, a. line 141. "m_ij" -> "h" b. In the caption of table1, "VBPI-GNNuse" -> "VBPI-GNN use" c. line 213, "3.1 Experiment Setup" -> "4.1 Experiment Setup"

Questions

1. Could you clarify whether the computation of the marginal log likelihood involves an evolutionary model like [1]? Does the evolutionary model utilize aligned sequences? If so, does this contradict the statement in the article that sequence alignment is not required? 2. I noticed that the author provides a comparison of run times. Could you provide details about the GPU used in the experiments, as well as the memory utilization? [1] M. Zhou, Z. Yan, E. Layne, N. Malkin, D. Zhang, M. Jain, M. Blanchette, and Y. Bengio. Phy-logfn: Phylogenetic inference with generative flow networks. arXiv preprint arXiv:2310.08774, 2023.

Rating

7

Confidence

4

Soundness

2

Presentation

3

Contribution

2

Limitations

I have no concerns on the potential societal impact of their work.

Summary

The paper propose phylogenetic tree inference by modeling it as a problem of conditional-constrained tree structure generation. Its goal is to jointly generate and optimize the tree topology and branch lengths. By mapping species sequences into a continuous geometric space, PhyloGen performs end-to-end variational inference without limiting the possible topologies. To maintain the topology-invariance of phylogenetic trees, distance constraints are applied in the latent space to preserve translational and rotational invariance. They have shown the proposed model is effective across eight real-world benchmark datasets.

Strengths

The paper addresses very interesting problem and propose novel approches.

Weaknesses

I am not sure if the formulation equation 7, adding the term R mathematically make sense, as orignal ELBO in equation 6, is the lower bound, now adding the term log of R which is negative, and trying to maximize equation 7 would result in minimizing R? Also in equation 7, q(\tao (z)) seems missing, or is there a kl between R and Q from module B? I think the paper presentation needs to be improved. The figure 2 is very helpful but the text and some notations are a bit confusing.

Questions

1. Regarding the statement, "Existing Variational Inference methods, which require pre-generated topologies and typi6 cally treat tree structures and branch lengths independently, may overlook critical sequence features, limiting their accuracy and flexibility", can you elaborate on what kind of features do you refer to that the method could overlook? 2. I am not sure fromt he distance matix to the tree topology, the Neighbor-Joining (NJ) algorithm can be differentiated to be able to learm end to end? 3. I think the section C.1. Topology Learning is written somehow confusing. Especially from line 121 to line 124. Why the distribution p(\tao(z)) is represented as the distribution of y conditioned on tree structure? same as the q(\tao(z), B_\tao) is not clear what distribution it represents. 4. I am not sure in this kind of tasks, if the ELBO is a good metric to reflect the inference power of the model. 5. The VAE model basically takes the embedding of the genomics sequences, and the initial topology of the tree \tao, then try to reconstruct the tree (without the leaf length)? I was thinking what happens if we remove the learning of the first component, make the phylo tree construction constant using directly the embeddings from LLM? 6. I am not sure if I understood how the scoring function is implemented, f(z), z is the embedding of the genomic sequence, f applied on the leaf nodes? or all the nodes, and why minimize it?

Rating

5

Confidence

1

Soundness

2

Presentation

2

Contribution

2

Limitations

1. While PhyloGen outperforms many existing methods, it still requires substantial computational resources, particularly during the training phase. The complexity of jointly optimizing tree topology and branch lengths, along with maintaining topological invariance, sounds computationally heavy, for instance: 1.1 Monte Carlo simulations to sample from the posterior distributions of tree topologies and branch lengths. 1.2 The tree construction module computes distance matrices from latent variables and uses algorithms like Neighbor-Joining (NJ) to generate initial tree structures. 1.3 The utilization of a pre-trained genome language model (DNABERT2) to transform DNA sequences into genomic embeddings, 2, Looking at the table 5, I am wondering how it works in inference time, would peformance drops when we have less or more specious than the one in the training data the model was trained on? Also how it translates to other specious? moreover, did you train one model per dataset, or is there a way to train all together?

Summary

The authors propose a new method, PhyloGEN, for phylogenetic inference. The method is able to perform end-to-end variational inference in order to jointly optimize the tree topology and the branch lengths. To achieve this, the authors propose using a pre-trained genomic language model to extract genome embeddings, and the embeddings are then used to form an initial topology which is refined iteratively using the proposed topology and branch length learning modules.

Strengths

- The paper makes a novel contribution to an important research topic. Unlike previous variational inference methods, the method does not require pre-generated topologies. - The incorporation of a pre-trained genomic language model to the method is an interesting contribution and is something that will likely be increasingly explored in the future. - The method is compared against a number of recent phylogenetic inference methods, and it achieves superior performance in terms of MLL and ELBO on standard benchmark datasets compared to the previous methods. In addition, experiments are performed to investigate the robustness of the proposed method and the diversity of the generated topologies, indicating that the method is consistent with the "gold standard" MrBayes approach. The authors also provide an ablation study to understand the impact of the various design choices in their method.

Weaknesses

- The methods section (Section 3) is difficult to read in that variables are introduced without leading sentences, some notation is undefined, and design choices are introduced without clear motivation. - What is the motivation for $f_i$ and how is it defined in (2) if there are multiple children $f_j$? - The function $h$ is used before its definition. - Which function $h$ is the latter $h$ in the definition $h(x_i, x_j) = h(x_i, x_i - x_j, d_{ij}^2)$? - Why is max aggregation a good choice? - The scoring function $S$ is not described in enough detail. What does it output and how does it provide additional gradient information? - Most experiments, in particular RQ2, RQ4, RQ5, RQ6, and RQ7 are performed on only one data set (DS1), making it difficult to draw meaningful conclusions. I would also consider the runtime an important metric, but it is measured only in conjunction with the robustness assessment on only one data set. - The authors do not provide code for their method or experiments (although the code is promised to be released later). Without the code, considering the lack of detail in the methods section, it would be difficult to reproduce the proposed method. - Minor: - The red and yellow colors are opposite to what they should be in the caption of Table 1. - There is a subsection numbered 3.1 in Section 4. - RQ7 is only referred to in the Appendix, and not in the main text. - Typos: "TreeEnocoder" (Figure 2 caption), "Metrtic" (Figure 3 title)

Questions

- Could the authors explain that if MrBayes is considered the gold standard, and the proposed method is validated by comparing it against MrBayes in RQ2 and RQ3, why is the proposed method able to achieve such a big increase in MLL and ELBO while the other methods yield relatively similar results? Relatedly, do the resulting topologies look significantly different in comparison to the other methods? - The proposed method has low standard deviations in Table 2 for multiple datasets. Is this due to the optimization being robust with respect to the random seed or could it not indicate that $Q(z)$ does not support multiple topologies? Do the MrBayes posteriors support multiple topologies? - How does the Bipartition Frequency Distribution look for other methods like GeoPhy and ARTree? - How does the runtime of MrBayes compare to the runtime of the proposed method? - Can the authors hypothesize why a simple fully connected $S$ would work so well (Figure 3)? - In the construction of the distance matrix, could the authors explain why using XOR makes sense since aren't the $z_i^*$ continuous? Moreover, would it make sense to try out other similar algorithms to neighbor-joining to initialize the topology? - I would suggest that the authors move section 3.3 (with the derived gradients) to the Appendix, leaving space for a more detailed description of the various design choices.

Rating

6

Confidence

2

Soundness

3

Presentation

2

Contribution

3

Limitations

The authors do not explicitly discuss the limitations of their work. As I am not an expert in phylogenetic inference, it is difficult to judge the assumptions made in the paper. Are there conditions that are required to be met for using a pre-trained language model for extracting embeddings, and is the assumption of the tree topology and branch lengths being conditionally independent reasonable? It would be helpful for the authors to elaborate on what they think the possible limitations of their own work are.

Summary

This paper presents PhyloGen, a novel approach for phylogenetic tree inference using pre-trained genomic language models and graph structure generation. PhyloGen aims to jointly optimize tree topology and branch lengths without relying on evolutionary models or equal-length sequence constraints. The method demonstrates superior performance and robustness across multiple real-world datasets compared to existing approaches.

Strengths

Novel approach combining pre-trained genomic language models with graph structure generation for phylogenetic inference Joint optimization of tree topology and branch lengths without typical constraints Superior performance on benchmark datasets compared to existing methods Robust to data changes and noise Provides deeper insights into phylogenetic relationships Computationally efficient compared to baselines

Weaknesses

Limited discussion of potential limitations or failure cases Lack of comparison to some recent methods in the field Source code not provided

Questions

How does PhyloGen perform on larger datasets with hundreds or thousands of species? Have you explored using other types of pre-trained language models besides DNABERT2? How sensitive is the method to hyperparameter choices? What are the main computational bottlenecks of the approach? Have you tested PhyloGen on simulated datasets where the true phylogeny is known?

Rating

5

Confidence

2

Soundness

2

Presentation

2

Contribution

2

Limitations

While some robustness tests were performed (node additions and deletions), the paper doesn't explore all possible types of data noise or perturbations. The paper doesn't deeply explore the interpretability of the model's decisions or the learned representations. The paper doesn't extensively discuss how well the method generalizes to different types of genetic data or organisms not represented in the test datasets. The method relies on pre-trained genomic language models (specifically DNABERT2), but doesn't explore how performance might vary with different pre-trained models.

General response to all the reviewers

Dear Reviewers, We greatly appreciate your effort and valuable feedback. In our response, we carefully illustrated and answered all your questions in detail. Additional experiments and analysis results are also provided as you had requested. In addition, we further clarified several unclear statements in the paper. We have incorporated all changes into the revised manuscript for your consideration. We hope your concerns have been addressed. As you may know, unlike previous years, the discussion period this year can only last until August 13, and we are gradually approaching this deadline. We take it seriously and would like to discuss this with you during this time. We would be happy to provide more information based on your feedback or further questions. If you are satisfied with our response, please consider updating your score. If you need any clarification, please feel free to contact us. Best regards, Authors

Reviewers comment

Thank you for authors answer to clarify my questions. After reading the answer, I decide to keep my score, I think the idea is interesting but the paper presentations needs to be improved and parts of the methodology as well as the evaluation metrics needs to be clarified.