Summary
The paper's outstanding qualities lie in its well-articulated presentation and its precise experimental design. It amalgamates the earlier research findings of [Zeng et al., 2021] and [Hızlı et al., 2022] with the innovative notions put forth by [Robins et al., 2022] on indirect effects. The authors tackle pragmatic issues, such as the effects of surgery on a patient's blood sugar levels in relation to their diet, by formulating pertinent questions. For instance, they question whether optimal post-surgery mediation can entirely regulate blood sugar levels, or if there exist uncontrollable surgery-induced effects on blood sugar levels that resist management through mediation and diet adjustments.
In their methodological approach, the authors utilize non-parametric models of the temporal data-generating process. They employ a Marked Point Process (MPP) akin to [Hızlı et al., 2022] for meal intake (the mediating factor) and a non-parametric Gaussian Process (GP) for the outcome. The former is modeled as a combination of a counting process (number of meals) and a dosage process (carb intake per meal) using a non-parametric Poisson Process and the latter as a Gaussian process. The mediator model is trained to predict the mediator based on the intervention and the outcome model is trained with both the mediator and intervention given as input. This approach is put to the test in a series of synthetic experiments, validating its predictive strength against baseline models like Zeng et al., 2021. They further apply it in a real-world setting, successfully reproducing biological insights and potentially addressing the question of the degree to which surgery impacts diet changes.
While at first glance, this paper might seem to bear similarities to [Hızlı et al., 2022], it stands out through its adept combination of the best methods for examining direct and indirect cause-and-effect relationships in a temporal setting. The paper, with its organized code and clear writing style, has the potential to become a valuable asset. However, I'm leaning towards accepting this paper on the condition that certain concerns regarding its originality and novelty are addressed, which I will detail in the following.
Strengths
1. The paper is commendable for its realistic problem setup, which is articulated in Section 5.1 dealing with corner cases where the assumptions might falter: the existence of hidden confoundings and the violation of assumptions A.1 to A.3. Such validation is crucial in causal studies to confirm assumptions and identify any unnoticed confoundings that may influence our conclusions. Furthermore, the experiments and predictions are consistent with the clinically significant direct and indirect impacts of bariatric surgery on blood glucose levels.
2. The approach to modeling the temporal dynamic is robust, anchoring its foundation on recent, proven work that adds to its credibility.
3. The paper's eloquent presentation is worthy of note. The reading experience is enhanced by effective use of color-coding to differentiate between mediator and direct interventions. A minor suggestion would be to consider adaptations for grayscale printed versions of the paper. For instance, the caption of Figure (2) includes light and dark blue color coding, which could be made more distinguishable by slightly altering the arrow patterns.
Weaknesses
1. The theoretical advancement of the study appears relatively marginal. While the exploration of direct vs. indirect causal effects in a temporal setting is engaging and the experiments provide valuable insights, I have some reservations about two of the claimed main contributions:
* Dynamic causal mediation with a point process mediator: [Hızlı et al., 2022] have previously introduced point process mediator modeling. The novelty here is questionable, given that in the prior work, the treatment was the mediator itself.
* A mediator-outcome model with an external intervention: The distinctiveness here is the training of two models: pre-intervention and post-intervention. However, the applied intervention is overly simplistic, offering limited theoretical innovation or insight. I have proposed, in the "Questions" section, the inclusion of the theory behind more complex interventions and experimentation on the simpler case. Yet, as it stands, this contribution mainly replicates the approach from [Hızlı et al., 2022], but uses two models to account for the intervention.
2. Table 1 presents results suggesting that the direct causal impact of surgery outweighs its indirect effects. Although the insights from 5.1.3 and 5.1.2 align with existing studies, there seems to be no supportive evidence for this hypothesis. Perhaps incorporating relevant literature explanations into the discussion would be beneficial. While the coherence between findings in 5.1.2 and 5.1.3 lend some validation to the model, it would still be advantageous to have literature support for 5.1.4.
Questions
1. Even though the paper presents a succinct and coherent narrative, it's hard to ignore that the methodology could easily extend to cases where the intervention itself is also a point process. For instance, one might consider a patient's long-term history and periodic clinical treatments. In such scenarios, $NIE$ and $NDE$ could be defined at different time points. It might be beneficial to include the theory behind this in the appendix section. The theoretical framework in sections 2 and 3 would work if one defines $N_A: [0, T] \to \mathbb{N}$, and instead of developing two distinct models, a more comprehensive model could be formulated that includes the history of interventions. This approach could also minimize the chance of future incremental papers being published.
2. What is the model's predictive power under model misspecification? Currently, the paper only presents results assessing predictive power in semi-synthetic scenarios where the model is appropriately specified. However, it would be beneficial to conduct experiments in scenarios reflective of real-world settings where model misspecification is common. Although the paper does incorporate a real-world setting, it is used merely for extracting qualitative observations. A comparative analysis of prediction power, similar to the semi-synthetic scenario but including model misspecification, is missing.
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Limitations
1. Like any causality-oriented study, this paper relies on certain assumptions for its model to function effectively. However, the authors have acknowledged this constraint adequately. They have also indicated possible real-world scenarios with hidden confounders where these assumptions might be compromised. This demonstrates a good understanding of the model's limitations.
2. A further limitation of the method, as previously hinted, is its inability to handle more complex interventions. Currently, the model considers total effect, direct, and indirect effect only in relation to a unitary, binary intervention point process. It could potentially be enhanced by extending its capacity to accommodate non-binary scenarios (for different intervention styles, such as dosage treatments) or non-unitary processes (considering the entire electronic health record of a patient over a long period), which could provide a more comprehensive understanding.