Summary
This paper consider how to use data from past interventions to allow it to generalize to new unseen interventions. This is an important practical problem to consider, as running additional experiments is often costly/infeasible. In order to tackle this problem, the authors consider a graphical models approach, specifically they consider an interventional factor graph model (IFM). Using posit an IFM factorization of the the density $p(x; \sigma)$. Then they provide the sufficient conditions for identification, and provide a message passing algorithm to do so. Next, they discuss multiple approaches to estimate the density based on ML models, as as well via IPW methods. They also consider a covariate shift regression approach. Further, they provide a conformal inference approach to establish coverage. Lastly, they perform a number of experiments to establish the empirical efficacy of their method.
Strengths
The paper tackles a very important problem, how can we use past interventions to generalize to new interventions. They propose an innovative IFM model and message passing algorithm to establish identification for this problem. Viewing this problem under this lens is an interesting one, and potentially of practical use. Further, I appreciate the authors providing examples so that its easier to understand their identification argument. I also appreciated the authors providing multiple methods for estimating the density discussed earlier. I believe providing multiple approaches is often of great practical use since no one algorithm often works in all scenarios. In terms of empirical evaluation, I think its interesting that the authors used semi-synthetic data. I believe this is good practice, and should be followed more often.
Weaknesses
Presentation: Presentation of both the regression and coverage algorithm is confusing, and seems to require a lot of additional knowledge on behalf of the reader. I do not fully understand how these algorithms proceed. For example, the deep-energy based models, lines 237-239 are very unclear. Making it clear what exactly is being fit would be very useful. More generally, being clear and rigorous regarding these things will go a long way in making the paper clearer.
Empirical Evaluation of Coverage: I did not see any simulations to this effect.
Empirical Evaluation: It is not clear to me what exactly X is in these datasets. Once again, being clear and rigorous about these details will enhance understanding, and give the reader a chance to appreciate the empirical evaluation. Further, even after reading the appendix, I do not understand how the outcomes were generated. Could the authors please clarify empirical details in the rebuttal?
Comparison to related work: The comparison to [2] is incorrect. The authors claim that a series of works including that of [2] requite data to be collected for all regimes in $\Sigma_{\text{test}}$. This is not the case. For example, the experimental design section of [2] shows that this isn't the case.
Questions
I have made some suggestions in the weaknesses section. I list some other questions here.
Deep-energy based models: Does fitting the parameter vector $\theta_{k, \sigma_{F_{k}}}$ require knowledge of the set of variables in $F_k$? I am confused by this, and if it does require, how do we determine these variables in practice.
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Limitations
Yes, they have.