Causal Structure Learning Supervised by Large Language Model

Summary

This paper proposes to use LLM to assist causal discovery. Specifically, an iterative procedure that encodes LLM queries into score-based methods is used. Different from previous methods that use LLM for causal discovery, this proposed method is claimed to be able to handle the false discoveries by non-distinguishment between direct causal edges and indirect ancestral causal relations. The efficacy of the proposed method is validated empirically.

Strengths

1. The use of LLM in causal discovery is trendy and interesting. 2. The paper raises a crucial concern about distinguishing between direct causal edges and indirect ancestral causal relations in LLM-based causal discovery methods, which is a commendable effort. 3. The analyses on various significant concerns (section 4) are intriguing.

Weaknesses

1. **rst and foremost, I have reservations about using LLM in causal discovery methods:** - Causal discovery aims to recover the causal relations among observational **variables**, not **variable names**. When the causal relations are already known by people (otherwise it couldn't be answered by LLM), there seems to be little need for causal discovery on the data. - Therefore, utilizing LLM to assist in causal discovery, as in this paper to incorporate prior knowledge into the process, appears completely irrelevant to the core of either of LLM or causal discovery. The application of LLM is just another way to obtain the prior knowledge (similar as asking human experts/human-in-loop methods). - However, in terms of causal discovery with prior knowledge, there exists many prior work with solid theoretical foundations (e.g., algorithms, characterizations, https://arxiv.org/pdf/1302.4972.pdf, https://arxiv.org/pdf/1910.02997.pdf, https://arxiv.org/abs/2207.05067). This paper has none of the theoretical guarantee. - Also, this paper refers to "LLM supervised causal discovery." To clarify, there exists another paradigm of "supervised causal discovery (SCL)" (e.g., https://arxiv.org/pdf/2110.00637.pdf, https://arxiv.org/pdf/2204.04875.pdf) that employs data supervision to improve statistical estimations with theoretical guarantees. This paper does not belong to that category either and should be termed "LLM facilitated/assisted causal discovery" to distinguish it. 2. **There are notable theoretical flaws in this paper:** - As mentioned earlier, the proposed algorithm is just greedy and seemingly intuitive, but lacking theoretical guarantees. For example, + Why does the algorithm apply LLM-answered causal prior knowledge only to score-based algorithm's output of existing edges and not missing edges? Although section 4.1 touches on this, it remains relatively vague and lacks formal proofs. + When LLM answers "B.changing V2 causes a change in V1," why does the algorithm add a direct edge V2->V1 into prior knowledge? It should represent an ancestral relation. + Similarly, when the LLM answers "C.changes in V1 and in V2 are not correlated.", why the algorithm only eliminates the direct edges between V1 and V2? It should be all the ancestral relationships got eliminated? + Similarly, when LLM answers "C.changes in V1 and in V2 are not correlated," why does the algorithm only eliminate direct edges between V1 and V2? It should eliminate all ancestral relationships. + The listed three prior constraints: the second one and the path existence one are equivalent, not just a one-way implication. When Xi precedes Xj in every causal ordering consistent with the DAG, Xi must be an ancestor of Xj. + The authors only trigger the LLM query for directed edges predicted by score-based algorithms. However, nonparametric score-based methods typically identify only a PDAG. How do they handle undirected edges, or what parametric assumptions are required if they assume DAGs can be output? + How to demonstrate that the algorithm can terminate (i.e., Line 13 of Algorithm 1, "until no new constraints are added")? Furthermore (and more practically), how to identify and resolve conflicts between answers provided by LLM and conflicts within the data distribution? - Overall, I recommend the authors to disentangle this work from LLM and view the proposed algorithm as causal discovery with oracle prior knowledge. They should then evaluate and establish the correctness of the algorithm within the context of causal discovery with prior knowledge.

Questions

See "Weaknesses" part.

Rating

1: strong reject

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

1 poor

Presentation

2 fair

Contribution

1 poor

Summary

The authors give a method for using LLMs to provide expert knowledge to help guide a discrete score search, attempting to take advantage of the strengths of LLMs and CSLs. The idea is that the expert knowledge obtained from the LLM can help to guide the discrete score search by updating the priors of the scores used in the search.

Strengths

Interfacing LLMs with CSL is an area of current interest, so I'm glad the authors are making positive proposals in this direction.

Weaknesses

I had a number of issues, as follows: 1. The style of the paper was a little hard to read; perhaps another couple of rewrites concentrating on clarity might help. 2. Here are a couple of papers on this topic that I’m familiar with; perhaps these could be cited and discussed: Kıcıman, E., Ness, R., Sharma, A., & Tan, C. (2023). Causal reasoning and large language models: Opening a new frontier for causality. arXiv preprint arXiv:2305.00050 Jin et al. paper: Jin, Z., Liu, J., Lyu, Z., Poff, S., Sachan, M., Mihalcea, R., ... & Schölkopf, B. (2023) 3. The first of these papers discusses the idea of using LLMs to obtain expert background knowledge for causal search and also discusses the possibility of inferring causal structure from metadata. Also, the need to check whether the LLM is simply regurgitating memorized facts instead of inferring causal structure is discussed (cf. their Tubingen pairs analysis). The second paper gives negative results for inferring causality from data alone using LLMs. Both of these seem relevant to the discussion in this paper. 4. I actually recognized the particular way in which the authors suggested for including background knowledge in a CSL search—by updating the priors for the various scorings done in the search based on known edges in the graph. The last time I saw this approach used in a paper was in: Sachs, K., Perez, O., Pe'er, D., Lauffenburger, D. A., & Nolan, G. P. (2005). Causal protein-signaling networks derived from multiparameter single-cell data. Science, 308(5721), 523-529. So, this also is not a completely novel approach, at least on my reading of it (though please correct me if I’m wrong). 5. There are other ways to include background knowledge in a CSL search, as implemented, for instance, in the PCALG package or in the Tetrad package—where particular edges can be identified as forbidden or required in a search with rules applied for updating the graph structure given these constraints. A mention of this methodology might have proven useful, or better yet, a comparison to this approach. I don’t know how to incorporate knowledge of required or forbidden paths (longer than a single edge) using that technology though. 6. Minor point. Section 5, “causl” -> “causal.”

Questions

Is it the case that only discrete datasets (e.g. from Bayes Net Repository structures and elsewhere) are being targeted? I couldn’t quite tell from the description. If so, this could be stated up front for clarity. Can linear systems be addressed using this approach? Obviously not with BDeu or discrete BIC for instance. What about mixtures of continuous and discrete datasets? What about general distributions? Is there a reason for concentrating on discrete datasets, such as a particular application one is targeting?

Rating

3: reject, not good enough

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Summary

The paper proposes an algorithm (Algorithm 1) for causal structural learning with the help of large language model (LLM). In particular, LLM is iteratively used to discover constraints on structures as defined in (Li & Beek, 2018). The main contribution is the prompt in Sec. 3 (1), inspired by (Kiciman et al., 2023), which asks the LLM to choose from one of the constraints. Some experiments are done to show the results.

Strengths

The idea of using LLM to supervise the structural learning is interesting. A practical algorithm is proposed that integrates the LLM supervision into the structural learning algorithm. In particular, the specific prompt for discovering structural constraints is provided. Experiments are done using SoTA LLM.

Weaknesses

One issue of the proposed method is that it lacks theoretical guarantee. It is unclear whether the LLM supervision is accurate, as many LLMs suffer from hallucinations. Moreover, the analysis in Sec. 4 is vague. It seems that Lemma 1 is among the only theoretical discussions, but the presentation of this result is poor. The proof in the appendix is even harder to follow if not wrong. At least the theoretical problem is not clearly defined. As an experimental paper, the LLM used is not discussed, making it difficult to reproduce the results.

Questions

1. Which LLM is used? What is the training dataset? Why choosing this model? Does the training set affect the CSL results? 2. Are there any theoretical guarantee on whether the LLM supervision is beneficial at all? 3. What are the computational and memory costs? 4. Bayesian network is not properly defined. Eq (1) and (2) shouldn't serve as definitions. 5. In lemma 1, are there any assumptions on the score function? 6. In the proof of lemma 1, is it necessary that the score comes with regulation? 7. What are exactly the LLM-related parameters p above Eq (11)? 8. Did you try using other LLMs such as llama?

Rating

3: reject, not good enough

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

2 fair

Presentation

1 poor

Contribution

2 fair

Summary

This paper presents the iterative LLM supervised CSL framework, which uses LLMs to contrain CSL with knowledge-based causal inference and iteratively refines the causal DAG based on LLM feedback. In comparison to the available LLM-based CSL work, the ILS-CSL is capable to offer constraints on direct causality, and it is claimed to be able to reduce prior errors significantly while using LLM resources. The paper performs evaluation experiments on eight real-world datsets to show its dominant performance.

Strengths

The new idea of using LLM to provide directed edge (arc) information/constraints for CSL.

Weaknesses

1. The mathematics is not formal nor rigorous. The notations are not well defined or explained. Although we are familiar with causal inference and could more or less guess what these symbols mean, the notations are not so consistent through different papers and readers may not be able to guess correctly. And the mathematical description is in a style between strict math and pseudo-code-like language. So are the proofs in this paper. We strongly recommend the authors to re-write all mathematical propostations and reasoning in a strict and standard manner. 2. The paper needs to be well organized, where unnecessary technical points should be trimmed and the main presentation of ILS-CSL should be presented clearly. It is difficult for us to follow your main story of ILS-CSL. 3. Due to the informal mathematical presentation of the method and disscussions in Sec.4, it is fairly difficult for us to review what the authors claimed and reasoned. For example, in Sec.4.1, what it claimed as the consequence (the max problem) seems not following in nature. We strongly recommend the authors to learn and adopt the strict math language from Causality Inference and Graph Theory. Such an effort is worthy for better presentation of your contributions. 4. It is hard for us to understand why a general LLM model could always help to enhance the causality performance rather than deteriorate. We understand that it is most likely difficult to prove it in math. That's why we have to refer to your strict presentation of your method in math and your discussions of its properties to understand this point. It sounds reasonable if the LLM used in a particular CSL task has learned of the domain knowledge of that task. The LLM helps to contribute positively like an human expert for the task. However, in your experiments, it seems you are using a general LLM without fine-tuning; thus, why such a LLM helps to contribute positively to tasks from various fields, rather than behaves like a "layperson" to offer wrong prior information to misguide the whole CSL task.

Questions

1. In Sec.2, page 2, the subsection of "causal structure learning": The mathbf "D" is the data (as you defined), but D is defined in the set of m-by-n natural numbers? You are dealing with causal inference from data that are consists of natural numbers? We assume it is a typo and the data are of m-by-n real values. However, in later formulations, you placed D like nodes/vertices, eg. (1), it is confusing. And we don't think eq. (1) is correct, please refer to classic literature on causality. And you have to be careful with what you are dealing with, Markov networks (indirect) or Bayesian networks (direct), especially some subtle differences appearing in their definitions and properties. 2. In eq. (4), the two set minus is conducted like minus in algebra, from left to right in sequence? And in eq.(5), what is exactly the constraint "G in DAG" in this optimization? Is it a constraint that we have froce the graph to be DAG? It seems a natural language description, how could you rewrite it in math such that this contraint can be tackled in optimization? 3. In eq.(7), the P(lambda) is a hyper-parameter? If it is a parameter to be specified by users, it seems (7) can be split into two parts which are scaled by two scales, P(lambda), (1-P). It behaves like a regularization parameter for penalties for (8). However, on the other hand, P(lambda) has specific math meaning and the structure constraint lambda is varying from an empty set to some set in your algorithm. Why is it reasonale to assume its probabilty, P(lambda), to be a fixed value?

Rating

3: reject, not good enough

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

1 poor

Contribution

2 fair

Summary

This paper proposes a new method ILS-CSL to do causal structure learning (CSL) by using both data driven approach + LLM, where the metadata of the dataset (e.g., the meaningful variable names) is available. Different from previous approaches which utilize the pairwise ancestral information, ILS-CSL uses the data driven approach as driving force to let LLM decide the edge existence. The decided edge existences are used as either soft/hard constraint to feedback to data driven approach iteratively. By doing so, the paper shows the reduction of error counts produced by LLM, and ILS-CSL consistently improves the accuracy of CSL in 8 datasets from bnlearn repository.

Strengths

1. The targeted problem is valid and important, timely Since the year 2023, there has been more evidence that LLM has a sort of commonsense causal knowledge, and it is very important to consider leveraging its power to enhance data-driven CSL for causal discovery. 2. The idea of using the data-driven approach as a driving force to let LLM decide the edge existence. I appreciate the efforts on combining the both in a smart way. commonly. This idea is novel and its effectiveness is demonstrated. 3. The paper is well written. Section 4 (Analysis of important concerns) is also interesting in showing semi-quantitative comparison between this approach vs. ancestral estimation approach

Weaknesses

See my detailed questions below

Questions

1) How would the effectiveness change when the baseline data-driven method goes stronger? The used baseline data-driven approaches, such as MINOBSx or CaMML seems not the very strong ones, for example, their performance on even Cancer and Asia are too low (as far as I know, there are quite some methods can achieve near perfect performance). This seems to suggest that, the ILS-CSL has significant improvement when the baselines are weak? 2) dealing with the situations when the data driven approach give near empty graph Algorithm 1 shows that if the LLM would not help when the data driven approach outputs an empty graph. We know that there are hyper-parameters that would balance between FP and FN, so in some setting, the algorithm would tent to be conservative to produce edges. I wonder authors opinion on this. 3) The method seems can be extend to deal with continuous data naturally, why not conduct such experiment? 4) In Section 4.2 ESTIMATION OF PRIOR ERROR COUNTS, a) the number of pairwise nodes devoid of paths is \gamma N(N-1)/2, but these paths does not only have X-...->Y or Y-...->X, but the paths with forks. these cases do not fall into the "Extra Causality", so the E_full is an upperbound? b) also, I wonder how the ratio:=E_ours/E_full change when the performance of baseline data-driven method goes? e.g., if the baseline goes better, how will the ratio change?

Rating

6: marginally above the acceptance threshold

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Ethics concerns

One concern: since LLM infers causality based on metadata such as variable names. will, in this case, some improper or bias causal effects will be generated? e.g., Gender causes something?