Summary
This paper presents Exogenous Matching (EXOM), a new importance sampling method for estimating counterfactual probabilities in Structural Causal Models (SCMs). EXOM transforms variance minimization into a conditional distribution learning problem, providing an upper bound on counterfactual estimator variance as per Theorem 1. It outperforms existing methods across various SCM settings and integrates well with identifiable neural proxy SCMs for practical applications. By incorporating prior knowledge through Markov boundaries, EXOM further enhances performance, demonstrating its potential as an efficient tool for counterfactual estimation in diverse scenarios.
Strengths
1. EXOM provides a tractable and efficient approach for counterfactual estimation in general settings, including scenarios with discrete or continuous exogenous variables and various observations and interventions. This flexibility makes it applicable to a wide range of causal inference problems.
2. The method is built on solid theoretical grounds, with the authors deriving an optimizable variance upper bound for counterfactual estimators.
3. The authors incorporate structural prior knowledge, specifically Markov boundaries, into the neural networks used for parameter optimization. They empirically validate the effectiveness of this approach across various scenarios.
4. EXOM consistently outperforms other importance sampling methods in various SCM settings, as demonstrated by the experimental results. Its compatibility with identifiable neural proxy SCMs further enhances its practical applicability.
Weaknesses
1. Theorem 1 relies on the assumption that the density ratio $q(\mathbf{u}|\mathbf{y}_ {\ast})/q(\mathbf{u}|\mathbf{y}_ {\ast}^\prime)\leq \kappa$ holds for all $\mathbf{u} \in \Omega_{\mathbf{u}}$ and $y_{\ast}$, $y_ {\ast} ^\prime \in \Omega_{\mathbf{Y}_{\ast}}$. This assumption may be overly stringent, as probability measures with infinite support sets might easily violate it. Could the authors elaborate on this assumption and provide examples of distributions that satisfy it?
2. In the Sampling and Optimization section, the distribution of the exogenous variable $\mathbf{U}$ is assumed to be known. However, in practical scenarios, $\mathbf{U}$ is often unknown, necessitating additional efforts to estimate $\mathbf{P_U}$ [A]. Could the authors provide further clarification on this assumption and discuss potential methods for estimating $\mathbf{P_U}$?
3. I understand the authors only consider models that provide identifiability results. However, it is encouraged to include neural proxy SCM methods based on VAE and DDPM as experimental baselines. While these may lack identifiability guarantees, comparing against them would further illustrate the superiority of the proposed method in relation to current state-of-the-art techniques.
4. While the method shows good performance on the tested SCMs, it's unclear how well it scales to larger, more complex causal models. The experiments are conducted on relatively small SCMs, and scalability to high-dimensional or densely connected causal graphs isn't thoroughly addressed.
[A] Ren, Shaogang, and Xiaoning Qian. "Causal Bayesian Optimization via Exogenous Distribution Learning." *arXiv preprint arXiv:2402.02277* (2024).
Questions
1. In Table 1, the EXOM method with MAP shows significantly better performance on the SIMPSON-NLIN and NAPKIN datasets compared to EXOM with GMM, whereas the performance on the FAIRNESS-XW dataset is similar for both methods. Could the authors provide further explanation for this discrepancy? Why does the GMM approach underperform in these specific cases, and what factors contribute to the similar performance on FAIRNESS-XW?
2. In the ablation study investigating the impact of injecting Markov boundaries, could the authors please include the performance results of EXOM without Markov boundaries? This comparison would further illustrate the benefit of incorporating Markov boundaries.
Limitations
The authors discuss the limitations of their work in Section 6 and Section D.3. Notably, this work does not have a negative societal impact.