Summary
The paper introduces a generative modeling approach called Frugal Flows, designed to learn the data generation process with an explicit parametrization for the marginal causal effect of treatment on outcomes. Inspired by the frugal parametrization of marginal structural models, this approach models the marginal intervention distribution $p(Y|do(X))$ directly, rather than the joint distribution $p(Y|Z, do(X))$. This helps in preserving any constraints on the average treatment effect while flexibly modeling the data generation process. Frugal Flows employs copula flows to parameterize the model, accommodating constraints on the average causal effect and handling unobserved confounding during data generation. The authors validate the proposed method through experiments on both synthetic and real-world datasets, demonstrating its ability to generate realistic datasets with user-specified constraints.
Strengths
* The paper's approach to validating causal models using simulated datasets is indeed impactful and relevant. It addresses a significant gap by allowing for general constraints on quantities of interest, such as average causal effect and unobserved confounding, during data generation. This capability is crucial because many prior generative modeling approaches for causal datasets either do not offer such flexibility or cannot ensure the preservation of these constraints, thus making this work a notable advancement in the field.
* The paper is well-written, with clear explanations in the background sections on frugal parametrization and flows, which help the reader grasp the proposed approach. The details of the approach are well-articulated, and the experimental results are presented effectively.
* The proposed approach is indeed novel. While it builds on established concepts like frugal parametrization, the specific application of normalizing flows for parametrization and its focus on average causal effect estimation represent a significant and innovative contribution.
Weaknesses
My main concern with the work is the limited empirical validation of the proposed approach. Given that the primary contribution is the learning methodology rather than theoretical analysis, I would expect a more extensive set of experiments to validate its effectiveness. For example, prior research on generative modeling for causal inference, such as the work by [1], includes comprehensive experiments with various statistical tests to assess the realism of generated samples and a broader benchmarking of causal estimators. This paper would benefit from similar depth in its empirical evaluation.
It would nice if the authors can conduct similar experiments to asses whether learned generative model generates realistic samples and evaluate it on more datasets. Also, the authors should compare with the prior works [1, 2] as baselines to establish which approach is the best at capturing the underlying data generation process, and empirically validate their claim (Section 2.6) that the proposed approach would be better than prior works at capturing used-specified constraints on the average causal effect.
References
[1] Neal, Brady, Chin-Wei Huang, and Sunand Raghupathi. "Realcause: Realistic causal inference benchmarking." arXiv preprint arXiv:2011.15007 (2020).
[2] Harsh Parikh, Carlos Varjao, Louise Xu, and Eric Tchetgen Tchetgen. Validating causal inference methods. In International conference on machine learning, pages 17346–17358. PMLR, 2022.
Questions
* A suggestion for the notation is that the authors could use $T$ instead of $X$ to denote the treatment variables in the paper. This way the notation would be less confusing, as $X$ represents a general random variable in Section 2.5 and Section 2.6
* Maybe there is a typo in Figure 2? The top row should be $\mathcal{F_{Z_i}}^{-1}(.)$ as we transforming the covariate $\{ Z_i \}$ to the correlated uniform variables $\{ V_{i} \}$
* I don't understand the Section 3.1.1 on copula flow for $X$ on $Z$. Why don't we directly model $p(X|Z)$ using a normalizing flow and why do we need to parametrize using copula flow as $p(X|Z)= p(X).c(X|Z)$?
Limitations
Yes, the authors have adequately addressed the limitations of their work.