Causal Effect Identification in a Sub-Population with Latent Variables

Summary

This paper addresses the S-ID (Sub-population Identification) problem in causal inference, extending it to scenarios with latent variables. The S-ID problem seeks to determine if a causal effect in a specific sub-population can be uniquely computed from observational data pertaining to that sub-population. The authors introduce new graphical definitions such as S-components and S-Hedges, which are extensions of classical notions like C-components and Hedges. They present a sufficient graphical condition for determining if a causal effect is S-ID and propose a sound algorithm for solving the S-ID problem in the presence of latent variables. Additionally, they show a reduction from the S-Recoverability problem to the S-ID problem.

Strengths

Technical quality: The paper presents thorough theoretical analysis, including formal definitions, lemmas, examples and theorems, as well as the reduction derivation.

Weaknesses

Empirical evaluation: there is no experiments at all besides the last section in appendix briefly describling how the authors want to conduct them. That is to say, this paper lacks experimental results or real-world case studies to demonstrate the practical applicability and performance of the proposed solution, especiallly for the two recursive algorithms Comparison to other existing methods: The paper surely follows the id, c-id, S-Recoverability literature for related work, but still could benefit from a more extensive comparison with other latent variable models in causal inference, [1-3] to name a few [1] Liu, Yuhang, et al. "Identifying weight-variant latent causal models." arXiv preprint arXiv:2208.14153 (2022). [2] Sherman, Eli, and Ilya Shpitser. "Identification and estimation of causal effects from dependent data." Advances in neural information processing systems 31 (2018). [3] Kocaoglu, Murat, Karthikeyan Shanmugam, and Elias Bareinboim. "Experimental design for learning causal graphs with latent variables." Advances in Neural Information Processing Systems 30 (2017).

Questions

1. In Remark 5.3. the authors "conjects that this algorithm is also complete". Is there any example or situation that it returns a false negative? 2. The reduction from S-Recoverability to S-ID in Section 6 seems to suggest that S-ID is a more general problem. Is there any scenario where solving S-ID would be more useful than solving the S-Recoverability problem?

Rating

6

Confidence

3

Soundness

3

Presentation

2

Contribution

3

Limitations

The paper proved that both proposed algorithms are sound for s-id, but did not show any guarantee for the completeness. Nevertheless, this limitation is mentioned in conclusion.

Summary

This paper extends the sub-population causal effect identifiability (S-ID) problem to include latent variables by adapting classical graphical definitions such as connected-components and Hedges. It proposes a sound algorithm to compute causal effects in sub-populations with latent variables.

Strengths

1. The paper is written well and easy to understand. 2. Examples in each section helps to understand the underlying idea easily. 3. All the necessary background is discussed clearly.

Weaknesses

1. Example 1 could be a better one because socioeconomic status can cause cardiovascular disease. 2. It would be good to include a sub section for summarising any assumptions made. 3. It would be good to include some real-world use-cases benefiting from such setting of causal effect identification.

Questions

See weaknesses section

Rating

7

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

Limitations are discussed

Summary

This paper extends the S-ID problem, which asks if a causal effect within a specific sub-population can be identified using only observational data from that group. The authors consider the scenarios where some variables are latent. They provide a sufficient graphical condition to determine whether a causal effect is S-ID and propose an algorithm based on this criterion. While the paper proves the algorithm's soundness, it suggests it might also be complete. Finally, they show that solving S-ID can solve a related problem called S-Recoverability.

Strengths

- The paper addresses an important problem in causal inference. - The paper is very well-written and covers the prerequisites very well.

Weaknesses

See the Questions section below.

Questions

- Although the work provides rigorous theoretical contributions, it would have been nice to evaluate how it would also work empirically, especially in a close-to-real-world scenario. - There are many variables and notations used throughout the paper. A table summarizing these notations and their definitions could improve readability. - How often do real-world problems satisfy the restrictive condition in Equation (8)​​?

Rating

7

Confidence

2

Soundness

3

Presentation

3

Contribution

3

Limitations

The limitations of the proposed approach have not been discussed.

Summary

The paper presents a sound algorithm for checking the s-identifiability of causal effects under sub-populations. This work complements earlier work on s-ID by generalizing the causal graphs to allow hidden confounders (no causal sufficiency). Specifically, the paper introduces the notions of s-components and s-Hedge, in parallel to the classical notions of c-components and c-hedge, and derives theoretical results based on those. The main theorem (Theorem 5.1) summarizes the condition under which a causal effect is s-ID, and a detailed algorithm is also proposed for deriving an identifying formula. Moreover, a reduction from s-recoverability to s-ID is mentioned which provides yet another approach to solve the s-recoverability problem.

Strengths

- I think the problem is quite meaningful since selection bias can be common in the data collection process. - It is great that the paper not only introduces novel notions such as s-components and s-hedges but also thoroughly reviews the classical notions of c-components and c-hedges, so we can make a comparison. The lemmas and theorems are also in parallel (but different ) to the previous ones for classical identification in [Tian, Pearl], which makes these profound concepts easier to penetrate. - I found the examples helpful, especially Examples 4 and 6, in aiding my understanding of definitions. - In general, a hard work that contains valuable theoretical contributions.

Weaknesses

- It seems that the s-ID method is sound but not complete, but I guess the paper already contains enough contributions and the completeness part can always be the future work. - More intuitions can be provided on the difference between s-ID and ID at the end of page 2 - it would be helpful to provide a more intuitive explanation for Example 1 (in addition to an explanation based on identifying formulas) for readers to see the importance of the problem. - The definition of $Q[]$ seems to be ambiguous. In Section 2 last subsection, $Q[X]$ is defined as the interventional distribution $Q[X] := P_{x} (V \setminus X)$, but in Theorem 3.4 $Q[D]$ seems to mean $Pr_x(D)$. It may be helpful to clarify the formal definition of $Q[D]$.

Questions

- Are there any insights on the difference between having $P^s(V)$ vs. having $P(V)$ for identifiabilty? For example, would more variables become dependent so they now belong to the same s-components when collecting data under $P^s(V)$? - Is there any evidence (counterexample) proving that Algorithm 1 is not complete? - Regarding the reduction from s-recoverability to s-ID, I'm wondering if there is any impact of this reduction besides theoretical interests. For example, will the reduction-based approach for s-recoverability be more computationally efficient?

Rating

7

Confidence

3

Soundness

3

Presentation

4

Contribution

4

Limitations

OK.

Thank you for your response

I thank the authors for their response. I've read their response and I will stay with my score.