On the Complexity of Identification in Linear Structural Causal Models

Learning the unknown causal parameters of a linear structural causal model is a fundamental task in causal analysis. The task, known as the problem of identification, asks to estimate the parameters of the model from a combination of assumptions on the graphical structure of the model and observational data, represented as a non-causal covariance matrix. In this paper, we give a new sound and complete algorithm for generic identification which runs in polynomial space. By standard simulation results, this algorithm has exponential running time which vastly improves the state-of-the-art double exponential time method using a Gröbner basis approach. The paper also presents evidence that parameter identification is computationally hard in general. In particular, we prove, that the task asking whether, for a given feasible correlation matrix, there are exactly one or two or more parameter sets explaining the observed matrix, is hard for $\forall R$, the co-class of the existential theory of the reals. In particular, this problem is $coNP$-hard. To our best knowledge, this is the first hardness result for some notion of identifiability.

Paper

References (40)

Scroll for more · 28 remaining

Similar papers

Peer review

Reviewer Gx9Z5/10 · confidence 2/52024-07-08

Summary

The paper examines the computational complexity of Generic and Numerical Identifiability problems. It provides some reduction from these problems to R complete classes. Based on these results, the paper proposes an algorithm for Generic Identifiability, which improves the state-of-the-art double exponential time.

Strengths

1- The paper proposed new findings on complexity of different various causal identifiability problems. 2- The results demonstrate that these problems can be decided in PSPACE. 3- The papers provided a comprehensive literature review in introduction.

Weaknesses

1- The importance of results and their technical novelty are not clearly discussed. 2- The main part primarily comprises theorems proofs, which are not completely clear. It would be beneficial to move some parts to the appendix and include more intuition and discussion in the main part. 3- The paper lacks experimental results demonstrating the correctness of the proposed algorithm and the improvement in running time.

Questions

1- Is it possible to conduct an experiment to compare your algorithm with previous algorithms? 2- Could you elaborate on the importance of the results and the novelty of proofs? The reductions seem very natural. I recommend to revise Sections 4, 5, and 6 as the results are very compact and ambiguous. I could not follow very well.

Rating

5

Confidence

2

Soundness

3

Presentation

2

Contribution

2

Limitations

Yes.

Reviewer gLJ25/10 · confidence 2/52024-07-11

Summary

The submission studies the problems of generic parameter identification and numerical identification, both of which represent prominent tasks in causal analysis. The results include a PSPACE (i.e., single-exponential) algorithm for generic parameter identification, as well as ForallR-hardness and a single-exponential time algorithm for numerical identification. The submission also extends these results to the closely related setting of edge identifiability.

Strengths

The studied problems are computationally challenging and seem to be relevant for the NeurIPS community.

Weaknesses

-The restriction to recursive models comes abruptly and is not motivated or discussed in the submission at all. A number of questions are left completely unanswered, such as: Are recursive models common? Have they been studied before? Do they occur in practice? Do the results generalize beyond recursive models? Is there a specific reason one should restrict one's attention to these models? -The definition of the problems studied do not seem written in sufficiently accessible language and assume that readers are already familiar with several specialized concepts. As one example, "Zariski almost all" and "Zariski closure" is never defined. The submission would also be much more accessible if it included at least some high-level examples of problem instances and solutions. -In terms of writing, several of the results are established using only semi-formal language and without what I would consider sufficient rigor. For instance, the construction for the proof of Theorem 2 is described using inline examples and descriptions, but given the crucial importance of the construction for the proof I would have expected the core properties of the construction to be established and formalized via lemmas or claims. -The results seem to be obtained primarily by cleverly combining known techniques and results, with little new insights required. In particular, the main results in Sections 5 and 6 essentially follow via direct encodings into appropriate fragments of the Theory of the Reals. Minor comments: -row 48: "using alone Sigma" should be "using Sigma alone" -row 71: malformed sentence -row 301: "So we can check in..." should be "Hence, we can check in..." -row 340: "*the* generic identification problem". Also, missing full stop at the end of the footnote.

Questions

No additional questions, but the authors are welcome to respond to the specific concerns raised above.

Rating

5

Confidence

2

Soundness

3

Presentation

2

Contribution

3

Limitations

The limitations are adequately discussed.

Reviewer Q8jB6/10 · confidence 2/52024-07-11

Summary

This paper looks into the parameter identification problem in linear structural causal models using only observational data. Under the Gaussian linear SEMs, the paper provides a polynomial-space algorithm that runs in exponential time. The paper also provides the hardness of the problem.

Strengths

In my knowledge, the methods and resutls are new. The paper investigated this traditional problem from computational complexity theory's view, which seems interesting to involved in this problem. The paper linked this parameter identification problems to the NP/coNP and PSPACE and then provide hardness and bounds based on it. Through I could not entirely follow the steps in the proof, but the paper seems theoritically sound.

Weaknesses

- The algorithm has some assumptions about the model itself. For instance, the model assumes normal noises and (known) topological sorting. But this does not weaken the paper's contribution. - The algorithm needs access to an oracle. - The notations can be further explained to enhance the reading flow; maybe give a few sentence definitions when the notions first appear in the paper.

Questions

- There should be a constraint on the diagonal of matrices in PD$(m)$. - My major question is regarding the construct of $G$ in Section 4: - The left figure in Figure 3 has all $\lambda$ with subscript $r$. Isn't it $i$ for the left and $j$ for the right? - There seem to be some typos in equation (11) as $\lambda_{a_{\ell},i}$ appears in all three summations. Is this equation obtained by expanding equation (10)? - Can the authors explain Observation 4 a bit more? I am not quite sure why this is the case. - The paper mentioned the algorithms in multiple parts. Does it mean the algorithm starts from line 320? And does this algorithm learn the final parameters or only give generic identifiability? - I'd suggest having a link from Koiran’s algorithm to Appendix D.

Rating

6

Confidence

2

Soundness

4

Presentation

2

Contribution

3

Limitations

The authors state all assumptions.

Reviewer WjE67/10 · confidence 4/52024-07-12

Summary

This paper is a theoretical examination of the problem of parameter identifiability in structural equation models for mixed graphs (which has applications in causal modeling), significantly improving upon known upper bounds and establishing a hardness result for the problem's complexity.

Strengths

**Originality**: The paper has originality (by using the existential theory of the reals to study complexity of parameter identifiability in probalistic graphical models), and this originality leads to an interesting new perspective and good results on the problem of study. **Quality**: The paper is of high technical quality (other than a few presumed typos). **Clarity**: I find the proofs and technical writing to be rigorous and clear, and the positioning compared to previous complexity results is quite clear. **Significance**: The paper should have at least moderate impact in the sub-areas Causal Inference and Graphical Models (and high impact in the sub-sub-area Algebraic Statistics). Specifically, this paper established foundational complexity results that have wide relevance in causal modeling. Additionally, the paper leads to natural follow-up work in the form of implementing the presented theoretical algorithm and approaching similar identifiability questions for other model classes from the same perspective.

Weaknesses

The main weakness (other than those mentioned in the Questions and Limitations sections below) is the clarity of the non-technical writing. There is not much to guide the reader through the technical parts, so I worry non-experts will have too much difficulty to see the value of these theoretical results.

Questions

My main questions are about the "causal" aspect of the work as well as implications for other model classes. 1. Don't all these results hold for Gaussian recursive mixed graph models generally instead of specifically causal ones? In other words, I don't see how any of the results make use of causal assumptions (like Reichenbach's common cause assumption or the causal Markov assumption) or interventional settings. To be clear, I don't think this is a bad thing—I just want to make sure I'm not missing some assumptions that limit the results. 2. Do these results apply immediately to Gaussian DAG models? Or only the upper bound? Or is there some fundamental difference in the DAG setup? Now some technical questions: 3. In the equation on lines 33 and 129, shouldn't it be $\epsilon_j$ to match the $X_j$ on the left-hand side? 4. In lines 43,44 and again 154, 154, it's claimed that knowing the $\lambda$s is sufficient for recovering the $\omega$s, but one also needs to know/estimate $\Sigma$, right? While this is reasonably clear to an expert, it's not explicitly mentioned in those sentences. 5. In line 214, doesn't it rather reduce to $x_1 = \ldots = x_n = 0$? 6. Can the authors elaborate on the last sentence of the proof of Theorem 2, especially after "and" (lines 269, 270)? It could help to rephrase Corollary 1 in those terms or add an explicit Corollary 1'. 7. What is meant with "By standard simulation results,..." on lines 6, 7? I don't see any simulations results in the paper or supplement.

Rating

7

Confidence

4

Soundness

3

Presentation

2

Contribution

3

Limitations

Limitations (such as restriction to the linear Gaussian setting and only having theoretical results) are clear from a detailed reading, but they could (and should) be more explicitly mentioned as limitations of the work in, e.g., the Intro.

Reviewer ToDP6/10 · confidence 3/52024-07-14

Summary

The paper tries to attach an important theoretic problem: the complexity of identification in linear SEMs. The author proposed an algorithm for generic identification with exponential running time while previous the best algorithm is in double exponential running time.

Strengths

1. The paper is well organized and presented. For a reader who is not very familiar with the complexity theory, the writer make the conception such as $\forall \mathbb{R}$ easy to understand. 2. The over all idea to connect the identification with ETR and QUAD is good and easy to follow.

Weaknesses

Purely from complexity the current algorithm is better than state-of-the-art, however we do not have any knowledge about the constant computational overhead in the algorithm. When applying the theory in the paper to build a practical algorithm, this might also be important.

Questions

See weakness.

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

Yes

Reviewer WjE62024-08-08

Thanks for the comprehensive response! I'm satisfied with the answers/explanations. I would suggest trying to incorporate your answers to 1. and 2. into the main text of the paper (abstract, intro, discussion, as appropriate) to make it easier for the broader causality and ML communities to understand the importance of your results (which would help to address the main weakness mentioned in my original review).

Authorsrebuttal2024-08-10

Thanks for reading our paper so carefully and for the helpful feedback. We will follow your suggestion.

Reviewer Gx9Z2024-08-09

Thank you for your response. I have raised my evaluation; however, I remain uncertain about the paper's contribution, as I am not too much familiar with this area.

Area Chair 4eg62024-08-10

Dear Reviewers, The authors have submitted the point-to-point response. Thankfully, many reviewers are engaged in the discussion. Could ToDP and Q8jB please acknowledge that you have read the rebuttal? Are there still some unresolved issues? AC

Reviewer ToDP2024-08-12

Thanks for the rebuttal, I believe the rebuttal resolved my concern.

Reviewer Q8jB2024-08-12

Thanks to the authors for the all the responses. They address most of my concerns. While I'm not entirely confident in this field, I would like to raise my score to a 6. Also, I suggest that in revised versions, the author can use the 1 extra page to further elaborate on the definition and terms.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC