A versatile informative diffusion model for single-cell ATAC-seq data generation and analysis

Summary

The paper introduces a new diffusion model ATAC-diff for scATAC-seq data generation and analysis. ATAC-diff using a latent diffusion model which is conditioned on latent auxiliary models to encode latent variables, and integrate GMM as the latent prior to capture genetic information. This paper introduces a mutual information regularization to maintain the connection between observed and latent variables. Extensive experiments show that ATAC-diff outperforms state-of-the-art models in both data generation and analysis tasks.

Strengths

This paper introduces a uniform model that is applicable for multiple tasks in scATAC-seq analysis. The method section is clear and easy to follow, providing a clear explanation on auxiliary module, integration of GMM and mutual information regularization. The paper provides an in-depth theoretical analysis, helps the audience to better understand the method. The experiment covers a wide range of scATAC-seq analysis, including clustering, generation, denoising and imputation. The results are comprehensive and shows advantage of the new model.

Weaknesses

Multiple baseline performance on the benchmark dataset does not match what was reported in previous publications. For example, [1] shows much higher PCA performance on PBMC10k dataset. This discrepancy makes me concern about the reliability of the experiment results. If would be great to include a few benchmark dataset to help cross validate the model performance, for example Buenrostro2018. There are top baselines that are not included in the experiment, please consider add the comparison on the latest baseline as well, such as scBasset. [1] Yuan H, Kelley D R. scBasset: sequence-based modeling of single-cell ATAC-seq using convolutional neural networks[J]. Nature Methods, 2022, 19(9): 1088-1096.

Questions

1. Explain the discrepancy on baseline performance and cross validation with previous publications. 2. Include more benchmark dataset for better cross validation of the model performance, such as Buenrostro2018. 3. Add top baselines, such as scBasset into comparison.

Rating

4

Confidence

3

Soundness

2

Presentation

3

Contribution

3

Limitations

yes

Summary

This submission introduces ATAC-Diff, a conditional latent diffusion model for scATAC-seq data. ATAC-Diff incorporates a few components to a basic diffusion model, including a Gaussian Mixture Model as a semantic prior over the latent variables and mutual information as a regularizer. ATAC-Diff is benchmarked on several tasks: latent representation clustering, (contitional) data generation, and denoising/imputation. ATAC-Diff is shown to do better than PeakVI and SCALE (among other methods for clustering). [UPDATE 8/9]: Adjusting score from Reject to Accept due to clarifications.

Strengths

- It seems to be the first application of a diffusion model to atac-seq data represented as cellsxpeaks - ATAC-Diff is benchmaked against other generative models based on VAEs, including SCALE and PeakVI - Performance of ATAC-Diff seems to be better than previous methods on 3 tasks (i.e., cell type clustering, data generation, and denoising). - ATAC-Diff can generate unconditionally and conditionally with assistance from a VAE

Weaknesses

- presentation of ATAC-seq data is not clear. The cell x peak representation is a processed version that doesn't necessarily encapsulate the full ATAC-seq data. Clarity that they are not generating ATAC-seq data but rather processed ATAC-seq data in the form of cell x binary peaks is necessary. - Example: other approaches to analyze ATAC-seq data incorporate sequences, eg. chromBPnet and AI-TAC. - Weak evaluations. The evaluations of cell clustering are not clear. It is not clear how ground truth was determined, given it is likely given by another computational method. The metrics are not clear what they are and their strengths and limitations. The umap visualization is not reliable. The authors state that they observe EC1, EC2, EC3 from the Forebrain dataset are close proximity in latent space. But latent space can be highly warped, rendering any Euclidean distances in umap space not meaningful. - Generation quality task is confusing. - When is unconditional generation needed? Why is this a meaningful task? - Not clear how the metrics are calculated. What is the ground truth here? - Why are generated cells averaged? Shouldn't generation be assessed at the single-cell level? - The ATAC-Diff w.o con seems quite high. This affirms my concern that the evaluations might be weak. - The proposed components GMM and MI regularizer are not evaluated via an ablation study. Are they even needed? - This study treats ATAC-seq data as a cell x (binary) peaks representation, instead of fragment counts, which improves scATAC-seq analysis (see Martens et al, Nature Methods, 2023).

Questions

Questions are integrated within Weaknesses (above).

Rating

7

Confidence

3

Soundness

3

Presentation

2

Contribution

2

Limitations

This study seems interesting and their method ATAC-Diff may be an advance, but it is difficult to assess given the poor presentation. There are several weaknesses such as questionable ground truth within the benchmarks. Moreover, the task itself seems to be dated, focusing on cells x binary peaks. Moreover, the components of the proposed approach are not tested in any ablation study, bringing their purpose questionable. This approach seems to be a new method, but one of many in the growing universe of scATAC-seq data and it's not clear what impact this work will have due to these limitations.

Summary

Generating simulated scATAC-seq data is important for developing new methods and gaining a deeper understanding of the data. However, the simulation is challenging due to dropout and high noise in the data. Authors proposed a diffusion + VAE type of method to solve the problem. The general idea is first to use VAE to project original scATAC data to a lower embedding space. Then impose a diffusion process in the embedding space. The lower embedding space is a GMM rather than the classical isotropic Normal, which is the novelty part of the method. This configuration makes biological sense as cells can be grouped as different cell types. Due to the introduction of GMM distribution in the hidden GMM space, there exists complications in generalising the diffusion loss function. The authors have shown nice and solid derivations in the appendix. The method is then applied to three datasets on three different tasks and achieved comparable performance as SOTA methods. It seems that authors have provided a convincing solution to the research question. While this paper is clearly written and the general idea is relatively easy to follow, it will be nice if the author could help to answer the following questions. 1. In Eq. 11, what is the parametric form of q_\phi(z|x_0) 2. In Eq. 14, what is the actual meaning of conditional information y, could you list the parameters 3. Could you provide a concrete network architecture of your network in a supplementary figure, i.e. including the tensors and their dimensions? 4. In sec 4.4.1, what is the dropout rate distribution in your simulated data, are they similar to the real data? Depending on the answers, I may change my ratings in the future.

Strengths

Generating simulated scATAC-seq data is important for developing new methods and gaining a deeper understanding of the data. However, the simulation is challenging due to dropout and high noise in the data. Authors proposed a diffusion + VAE type of method to solve the problem. The general idea is first to use VAE to project original scATAC data to a lower embedding space. Then impose a diffusion process in the embedding space. The lower embedding space is a GMM rather than the classical isotropic Normal, which is the novelty part of the method. This configuration makes biological sense as cells can be grouped as different cell types. Due to the introduction of GMM distribution in the hidden GMM space, there exists complications in generalising the diffusion loss function. The authors have shown nice and solid derivations in the appendix. The method is then applied to three datasets on three different tasks and achieved comparable performance as SOTA methods. It seems that authors have provided a convincing solution to the research question. While this paper is clearly written and the general idea is relatively easy to follow, it will be nice if the author could help to answer the following questions. 1. In Eq. 11, what is the parametric form of q_\phi(z|x_0) 2. In Eq. 14, what is the actual meaning of conditional information y, could you list the parameters 3. Could you provide a concrete network architecture of your network in a supplementary figure, i.e. including the tensors and their dimensions? 4. In sec 4.4.1, what is the dropout rate distribution in your simulated data, are they similar to the real data? Depending on the answers, I may change my ratings in the future.

Weaknesses

Generating simulated scATAC-seq data is important for developing new methods and gaining a deeper understanding of the data. However, the simulation is challenging due to dropout and high noise in the data. Authors proposed a diffusion + VAE type of method to solve the problem. The general idea is first to use VAE to project original scATAC data to a lower embedding space. Then impose a diffusion process in the embedding space. The lower embedding space is a GMM rather than the classical isotropic Normal, which is the novelty part of the method. This configuration makes biological sense as cells can be grouped as different cell types. Due to the introduction of GMM distribution in the hidden GMM space, there exists complications in generalising the diffusion loss function. The authors have shown nice and solid derivations in the appendix. The method is then applied to three datasets on three different tasks and achieved comparable performance as SOTA methods. It seems that authors have provided a convincing solution to the research question. While this paper is clearly written and the general idea is relatively easy to follow, it will be nice if the author could help to answer the following questions. 1. In Eq. 11, what is the parametric form of q_\phi(z|x_0) 2. In Eq. 14, what is the actual meaning of conditional information y, could you list the parameters 3. Could you provide a concrete network architecture of your network in a supplementary figure, i.e. including the tensors and their dimensions? 4. In sec 4.4.1, what is the dropout rate distribution in your simulated data, are they similar to the real data? Depending on the answers, I may change my ratings in the future.

Questions

Generating simulated scATAC-seq data is important for developing new methods and gaining a deeper understanding of the data. However, the simulation is challenging due to dropout and high noise in the data. Authors proposed a diffusion + VAE type of method to solve the problem. The general idea is first to use VAE to project original scATAC data to a lower embedding space. Then impose a diffusion process in the embedding space. The lower embedding space is a GMM rather than the classical isotropic Normal, which is the novelty part of the method. This configuration makes biological sense as cells can be grouped as different cell types. Due to the introduction of GMM distribution in the hidden GMM space, there exists complications in generalising the diffusion loss function. The authors have shown nice and solid derivations in the appendix. The method is then applied to three datasets on three different tasks and achieved comparable performance as SOTA methods. It seems that authors have provided a convincing solution to the research question. While this paper is clearly written and the general idea is relatively easy to follow, it will be nice if the author could help to answer the following questions. 1. In Eq. 11, what is the parametric form of q_\phi(z|x_0) 2. In Eq. 14, what is the actual meaning of conditional information y, could you list the parameters 3. Could you provide a concrete network architecture of your network in a supplementary figure, i.e. including the tensors and their dimensions? 4. In sec 4.4.1, what is the dropout rate distribution in your simulated data, are they similar to the real data? Depending on the answers, I may change my ratings in the future.

Rating

6

Confidence

4

Soundness

3

Presentation

4

Contribution

3

Limitations

Generating simulated scATAC-seq data is important for developing new methods and gaining a deeper understanding of the data. However, the simulation is challenging due to dropout and high noise in the data. Authors proposed a diffusion + VAE type of method to solve the problem. The general idea is first to use VAE to project original scATAC data to a lower embedding space. Then impose a diffusion process in the embedding space. The lower embedding space is a GMM rather than the classical isotropic Normal, which is the novelty part of the method. This configuration makes biological sense as cells can be grouped as different cell types. Due to the introduction of GMM distribution in the hidden GMM space, there exists complications in generalising the diffusion loss function. The authors have shown nice and solid derivations in the appendix. The method is then applied to three datasets on three different tasks and achieved comparable performance as SOTA methods. It seems that authors have provided a convincing solution to the research question. While this paper is clearly written and the general idea is relatively easy to follow, it will be nice if the author could help to answer the following questions. 1. In Eq. 11, what is the parametric form of q_\phi(z|x_0) 2. In Eq. 14, what is the actual meaning of conditional information y, could you list the parameters 3. Could you provide a concrete network architecture of your network in a supplementary figure, i.e. including the tensors and their dimensions? 4. In sec 4.4.1, what is the dropout rate distribution in your simulated data, are they similar to the real data? Depending on the answers, I may change my ratings in the future.

satisfactory response.

The authors have addressed my concerns. I still don't agree with UMAP analysis as it can also warp local spaces. I will adjust my scores accordingly.

Inquiry about the feedback from the reviewers

Dear ACs, I am sorry to bother you. Could you kindly assist in verifying if the reviewers have received the rebuttal information? It has been six days since the rebuttal period concluded, and we only have two days left to discuss my response to their comments and their feedback on this manuscript. Your prompt assistance is greatly appreciated. I am looking forward to your response. Thank you for your help.