Author's response
We thank the reviewer for their encouraging assessment of the strengths of our paper and for their comments, which we address below.
> "I think the assumptions (1, 2) should be more clearly stated"
Agreed; we have updated the manuscript to place the assumptions in the main text.
> "homogenous drift and diffusion processes assumption may not be able to model some types of dynamics"
This is definitely an interesting point, which we now discuss in the conclusion of the revised manuscript. In a nutshell, with homogeneous drift and diffusion functions, SCOTCH cannot model non-stationary dynamics. For example, systems with obvious changepoints (e.g. climate science models) will not have time-homogenous drift/diffusion functions. Unfortunately, our theoretical results no longer hold if time-dependent graphs are allowed.
> "...the explanation of the improved performance of RHINO on the Netsim data"
| Method | AUROC |
| --- | --- |
| cMLP | 0.93 |
| Rhino+NoInst | 0.93 |
| Rhino+g | 0.974 |
| Rhino | 0.99 |
With regards to the performance of SCOTCH vs Rhino, we copy (part of) Table 3 of Gong et al. (2022) above, showing that Rhino can achieve almost perfect AUROC on the Netsim dataset when allowed to use instantaneous effects, even when restricted to Gaussian noise (Rhino+g), it can achieve 0.974. Both Rhino+NoInst (no instantaneous effects), and the Granger causality method cMLP (which cannot model instantaneous effects) can only achieve 0.93 AUROC, indicating the importance of instataneous effects on this dataset. As explained in Section 3.2, SCOTCH also does not incorporate instantaneous effects, so the performance is worse compared with Rhino. We also highlight the corresponding sections in the revised paper.
As for SCOTCH vs Rhino+NoInst with both complete and missing data, we can see that the performance is broadly similar and superior to other baselines. Although the performance of Rhino is slightly better than SCOTCH with missing data, we are not entirely sure why SCOTCH does not outperform Rhino+NoInst. We suspect it may be due to pecularities of the NetSim dataset, or the fact that Rhino+NoInst uses a more complex noise distribution transformed by a normalizing flow, whereas SCOTCH uses a Gaussian noise modulated by a diffusion matrix.
> "What is the effect of the sparsity prior on the structure? Can a different type of sparsity be used?"
The reason we need the sparsity prior is because in theory, a fully connected graph will always have the greatest capacity to fit data. The purpose of incorporating a sparsity term is to avoid false positive edges, i.e. spurious edges that may slightly improve likelihood but are not present in the underlying system. In real-world systems (e.g. DREAM3), we found this helped to more accurately predict the ground-truth graph.
L1-based sparsity priors (Lasso, group Lasso, adaptive group Lasso) are the most common type of prior used in structure learning, so we adopt this for our method. That said, there are other possibilities. For example, another prior sometimes used in DAG structure learning (Eggeling et al. 2019) assigns probability $p(G) \propto \prod_{d=1}^{D} \frac{1}{{D-1 \choose |pa(d)|}}$, where $pa(d)$ is the set of parents of the $d^{\textnormal{th}}$ variable. In theory we can use any graph prior that assigns positive probability to every graph. The L1 prior turns out to be particularly convenient (in combination with our variational graph posterior) as the KL-divergence in Eq. (14) is tractable.
(Eggeling et al. 2019) Ralf Eggeling, Jussi Viinikka, Aleksis Vuoksenmaa, Mikko Koivisto. On Structure Priors for Learning Bayesian Networks. AISTATS 2019.