Response to Comment by Reviewer Tp5P
We are very grateful to the reviewer for their response.
> I had misunderstood the scope of the paper indeed. I thought the Gaussian case was trivial and that the point was to reduce non-Gaussian examples to Gaussian ones as a first approximation (through Definition 2). Now I understand that even the Gaussian case is complex.
>
> I thought that for Gaussian distributions, the minimizer Q in Definition 1 would necessarily be Gaussian, and thus introducing Definition 2 was relevant only for non-Gaussian distributions (hence the misunderstanding on the scope).
The reviewer is absolutely correct that the Gaussian case is also non-trivial in general. They also correctly recognize that it is _not known_ whether the minimizer Q is guaranteed to be Gaussian if P_MXY is jointly Gaussian. Even for Gaussian P_MXY, this is only known in a few specific cases such as when M is scalar, or if P_MXY satisfies a condition shown in [12].
> If the conjecture is true, i.e. that minimizers Q in the Gaussian case are Gaussian indeed, then the two definitions are identical. If I understand well, the contribution of the paper in that case is not to bring a new definition, but to make explicit use of the knowledge that the minimizer is Gaussian, to help the optimization process, …
Indeed, we do _not_ provide a new definition. Rather, the central contribution of our paper is to develop and evaluate an efficient method to estimate the PID for Gaussian P_MXY, by restricting the space of the minimizer Q to Gaussian variables. As the reviewer states, this would result in the correct solution if our conjecture is true. By assuming Q to be Gaussian, we can make use of the properties of Gaussian distributions (e.g., closed-form expressions for mutual information) to develop an efficient projected gradient descent optimizer, and then check how well we perform (please see _”Evaluating the Gaussianity assumption”_ below).
> … while this was not done in previous papers. Am I right?
A previous paper by Venkatesh and Schamberg [12] used a similar technique of restricting Q to be Gaussian for computing a different PID definition called the delta-PID. Our work makes significant advances over [12], on multiple fronts:
1. We compute a different PID definition, the ~-PID, which satisfies better properties than the delta-PID used in [12]. Most importantly, the ~-PID satisfies a very fundamental property called **additivity** (please see our response to reviewer DDWz on why additivity is fundamental).
2. We examine several examples (Examples 5-10), increasing in complexity and dimensionality, to show that restricting Q to be Gaussian is reasonable, and allows us to recover the ground truth. [12] only tested this for the simplest case of Gaussians with scalar M, and fails in more complex cases (Example 8; Fig 2 in the paper).
3. Our method is faster (please see Fig 7 in the PDF attached to the overall rebuttal).
4. Our method is capable of computing higher dimensionalities (as described in our response to reviewer DDWz).
5. Finally, we consider the problem of _estimation_, i.e., computing the PID from a covariance matrix that is estimated from real data, and show how bias in PID estimates can be corrected. This is not addressed in [12].
These points are summarized in the overall rebuttal near the top of this page.
**Evaluating the Gaussianity assumption**
Since we do not have a guarantee that the minimizer Q is actually Gaussian for all Gaussian P_MXY, we test what the reviewer asked in their original review, but for _Gaussian_ P_MXY. That is, we test: “how realistic is this assumption [of restricting Q to be Gaussian], how far are real optimal Q_MXY from their Gaussian versions”. We consider a number of examples of increasing complexity (Examples 5-10), where the ground truth PID values are known (starting with scalar M [13], and then using the additivity property). We show that our method is able to recover the ground truth, proving that for all the examples we consider, the optimal Q _is_ in fact Gaussian, and hence our conjecture still stands.
Incidentally, our method is also applicable to non-Gaussian P_MXY, because it only relies on the covariance matrix, as noted by the reviewer in their original review. So we further tested whether it was reasonable to apply our method to Poisson distributions (now extended to more non-Gaussian distributions; see overall rebuttal). Here, we do not have ground truth, so we instead compared our results with another ~-PID estimator [20] (which works only for discrete distributions with limited support). We showed that our PID estimator with Q-restricted-to-be-Gaussian comes very close to the discrete estimator of [20] on a multivariate Poisson distribution P_MXY (Fig 5 in the paper). We then used this as a basis to apply our PID estimator to real neural data, and demonstrated its utility in providing new insights about interactions between brain regions (Fig 6 in the paper).