Additional Answer to Reviewer NoBU 1/2
> ***The situation with quadratic setup is the most tricky here, since it is known that GW problem admits multiple solutions and, thus, the optimization problem may differ at each step of training depending on the calculated discrete entropic GW plans.***
➤ We agree that the quadratic setup is the most tricky one, and we thank the reviewer for explicitly mention this issue again. As mentioned in the general response, we set out to investigate this further (Alg. 1, pdf), and results can be found in Fig. 2, pdf.
We consider two datasets:
30-dim. Gaussians (20 dim. quadratic + 10 dim. fused term): For both the source ($\mathcal{\mu}$) and the target ($\nu$) distribution we sample mean vectors from $\mathcal{U}([-1, 1])$ and construct the covariance matrices by sampling each entry of their square roots from $\mathcal{U}([0, 3])$.
30-dim. single-cell data (the one used for modality translation, but no train/test split): 20-dimensional PCA of the normalized ATAC data, 20-dimensional PCA projection of the gene expression data, and 10-dimensional PCA of VAE space (shared space).
To assess the stability of the solution of discrete EOT, we fix one sample $X_0 \sim \mu$ from the source distribution, and compute the variance of $\hat{\pi}(\cdot|X_0)[10:]$, i.e. of the incomparable space $\tilde{\mathcal{Y}}$ only, which is obtained from a discrete EOT solver with different samples in source and target. We do this for $\varepsilon=0.01$ and $\varepsilon=0.0001$ with the Sq. Euclidean cost. We consider:
- outer coupling: this serves as orientation.
- Gromov-Wasserstein (GW): We use $\tilde{\mathcal{X}}$ and $\tilde{\mathcal{Y}}$, i.e., we use the last 20 dimensions of both the source and the target distribution.
- Fused Gromov-Wasserstein (FGW): We use $\Omega \times \tilde{\mathcal{X}}$ and $\Omega \times \tilde{\mathcal{Y}}$, $\alpha=\frac{1}{11}$.
- Gromov-Wasserstein with geodesic cost (Geodesic): We use $\tilde{\mathcal{X}}$ and $\tilde{\mathcal{Y}}$ with the geodesic cost.
- Gromov-Wasserstein with initialisation (with init): We use $\tilde{\mathcal{X}}$ and $\tilde{\mathcal{Y}}$. We use the initialisation scheme as discussed in paper (Appendix A, E).
> ***However, the situation with the GW case is not clarified – according to the experiment description (Appendix E.3), the authors introduce some heuristics to “preserve the orientation of GW solution”***
➤ We visualise what we mean by "preservation of the orientation" of the GW solution in Fig. 1 in the provided pdf, where we transport a 1-dimensional Gaussian to a 1-dimensional Gaussian. The LHS of Fig. 1 shows the result obtained without initialisation, and the learnt coupling is clearly a mixture of solutions, while on the RHS of Fig. 1, we visualise that GENOT-Q learns a coupling which appears to be no mixture of solutions. While this is only an example in 1 dimension, the results in Fig. 2, pdf, highlight that using the proposed initialisation scheme stabilises the orientation of the GW solution also in high dimensions, for both the Gaussian and the single-cell experiments.
> ***[...] in the case of biological tasks it is not needed. (Still, as a non-expert in biology, I can not assess the validity of this claim.)***
➤ We thank the reviewer for pointing this out, apologize for this bold statement, and would like to correct ourselves and weaken the statement to: "In the case of single-cell data, choosing the right cost might help to overcome the need for the initialisation scheme to a certain degree." as can be seen from Fig. 2, pdf, where the geodesic cost seems to help make the solution more unique with $\varepsilon=0.0001$, as well as the results in Fig. 4, manuscript, LHS. Moreover, we will add the sentence "Additionally, most single-cell applications are a FGW problem (as opposed to a GW problem), and empirically, the fused term helps to make the set of solutions of the EOT coupling less diverse, as can be seen from Fig. 5, FIg. 19, and [provided pdf, Fig. 2].". Indeed, the only pure GW problem in common single-cell applications we are aware of is SCOT [Demetci+22]. As we realised the performance of both SCOT and GENOT-Q is poor, we introduced a novel way to include a fused term into the task of modality translation by constructing a joint space by creating a joint conditional VAE embedding from gene activity (source space) and gene expression (target space).
> ***However, it is not obvious for me to what extent this is a serious drawback hampering the applicability of the approach to biological tasks.***
➤ We hope to have clarified this with the additional experiments and the answers above.