Fast Scalable and Accurate Discovery of DAGs Using the Best Order Score Search and Grow-Shrink Trees

Learning graphical conditional independence structures is an important machine learning problem and a cornerstone of causal discovery. However, the accuracy and execution time of learning algorithms generally struggle to scale to problems with hundreds of highly connected variables—for instance, recovering brain networks from fMRI data. We introduce the best order score search (BOSS) and grow-shrink trees (GSTs) for learning directed acyclic graphs (DAGs) in this paradigm. BOSS greedily searches over permutations of variables, using GSTs to construct and score DAGs from permutations. GSTs efficiently cache scores to eliminate redundant calculations. BOSS achieves state-of-the-art performance in accuracy and execution time, comparing favorably to a variety of combinatorial and gradient-based learning algorithms under a broad range of conditions. To demonstrate its practicality, we apply BOSS to two sets of resting-state fMRI data: simulated data with pseudo-empirical noise distributions derived from randomized empirical fMRI cortical signals and clinical data from 3T fMRI scans processed into cortical parcels. BOSS is available for use within the TETRAD project which includes Python and R wrappers.

Paper

References (30)

Scroll for more · 18 remaining

Similar papers

Peer review

Reviewer oAsg4/10 · confidence 2/52023-07-02

Summary

This paper introduces the best order score search (BOSS) algorithm and grow-shrink trees (GSTs) for learning directed acyclic graphs (DAGs) in machine learning and causal discovery. BOSS achieves state-of-the-art performance in accuracy and execution time, making it a valuable tool for problems with highly connected variables. The paper also applies BOSS to resting-state fMRI data, demonstrating its practicality and effectiveness. The algorithm is designed to be a more efficient alternative to the existing Greedy Sparsest Permutation (GRaSP) algorithm.

Strengths

- The paper introduces the concept of Grow-Shrink Trees (GSTs), which are tree data structures for caching the results of the grow and shrink subroutines of GS. This data structure is compatible with many permutation-based structure learning algorithms, including BOSS and GRaSP. The use of GSTs is a novel approach that efficiently stores information needed for running GS. - The paper is well-written and clear. It provides detailed explanations and examples, making it easy to understand the proposed algorithm and its components. The use of figures and tables also helps to illustrate the concepts and results. - The pape compares BOSS against other algorithms such as GRaSP, fGES, PC, and DAGMA on linear Gaussian data generated from Erd˝os-Rényi and scale-free networks. The results show that BOSS maintains a high level of accuracy while scaling much better than GRaSP, indcating that it could be a valuable tool in the field of structure learning.

Weaknesses

- One potential weakness is the lack of discussion on the limitations of the BOSS algorithm and GSTs. While the paper acknowledges that there is room for additional improvements, it does not provide a detailed analysis of the limitations of the proposed approach. For example, the paper could discuss the assumptions made by BOSS, such as causal sufficiency, and how these assumptions may affect the accuracy and scalability of the algorithm. Additionally, the paper could explore the applicability of BOSS to other types of data beyond fMRI, such as financial data or electronic health records. Addressing these limitations would provide a more comprehensive understanding of the strengths and weaknesses of the proposed approach. - Validation on more Real-World Data: The validation of the algorithm on fMRI data is a good step, but more extensive validation on diverse real-world datasets could strengthen the robustness of the findings.

Questions

- How does the BOSS algorithm handle the problem of confounding variables in fMRI data, and how does it ensure that the recovered causal networks are not biased by these variables? - Would there be non-linear causal relationships in fMRI data, and how does it ensure that the recovered causal networks capture these relationships accurately? - How does the performance of the BOSS algorithm vary with the level of sparsity in the underlying causal graph in fMRI data? - Would the performance of the model vary with the level of noise in data? - Discussion on Limitations: Could you include a section discussing the limitations of your work and potential areas for future research? - Results Interpretation: Could you provide more insight into how you interpreted your results and why you drew the conclusions you did? This would help to understand the implications of your findings.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

- The authors have discussed the limitations of their work in the context of the performance of the BOSS algorithm. They have compared it with other algorithms like GRaSP, fGES, PC, and DAGMA on linear Gaussian data and have also tested it on non-Gaussian data. They have acknowledged that while BOSS performs well in terms of BIC score and running time, its recall may be low in certain scenarios. Question: You have mentioned that BOSS has a low recall in certain scenarios. Could you elaborate on these scenarios? How could this limitation be addressed in future work? - It would be beneficial to include a section in your paper discussing the ethical implications of your work. This could include potential misuse of your algorithm, biases in the data that could affect the algorithm's performance, and the implications of incorrect predictions by the algorithm.

Reviewer oAsg2023-08-21

Response to Rebuttal

Thanks for the authors for the additonal explanation. I believe adding those details can further help readers better understand the work. I would like to remain my original ratings.

Reviewer P8PW5/10 · confidence 1/52023-07-03

Summary

It's beyond my knowledge so please disregard my scores and reviews.

Strengths

It's beyond my knowledge so please disregard my scores and reviews.

Weaknesses

It's beyond my knowledge so please disregard my scores and reviews.

Questions

It's beyond my knowledge so please disregard my scores and reviews.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

1: Your assessment is an educated guess. The submission is not in your area or the submission was difficult to understand. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

It's beyond my knowledge so please disregard my scores and reviews.

Reviewer yEGg6/10 · confidence 2/52023-07-06

Summary

The authors introduce two novel methods for learning Directed Acyclic Graphs (ADGs): best order score search or BOSS and grow-shrink trees or GSTs. These methods have a similar performance to state-of-the-art approaches (namely GRaSP and also others: fGES, DAGMA, LiNGAM, etc), but the authors proof that are less computationally intensive and therefore more scalable. The authors then validate their methods by learning brain networks from synthetic and real brain recordings (fMRI).

Strengths

- originality: the paper introduces two novel approaches for learning DAGs. Even though this work closely follows previous studies on permutation-based structure learning, it also presents significant new departures from the previous algorithms. - quality: the theoretical proofs seem solid and the methods are backed with a reasonable amount of numerical and experimental evidence. - clarity: the paper is very concisely and clearly written. Although a lot of the information is in the supplements. - significance: learning network from brain recordings is a very useful tool to understanding the brain and its pathologies. A method that significantly increases scalability with same accuracy as the state-of-the-art is therefore noteworthy.

Weaknesses

- Validation on real fMRI is (somewhat understandably) weak because the scale free connectivity of the brain remains a theory rather than a proven fact. - Many details of the work are left out from the main paper and included in the supplemental material. It is almost hard to understand the main paper without reading the supplements.

Questions

No questions.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Authors have adequately addressed the limitations of their work.

Reviewer 9NHG5/10 · confidence 1/52023-07-27

Summary

The paper proposes a computationally efficient algorithm using the grow-shrink trees to iterate through some of the combinatoric number of directed acyclic graphs in a graphical model. The approach builds on early work but is faster.

Strengths

The paper is straightforward, easy to read (examples are given), and focused in its presentation. The results show that it is faster than the baseline GRaSP.

Weaknesses

The paper's focus on the approach also means there is not much perspective given until the final discussion. The paper's claim of scaling to thousands is not thoroughly tested (results for 1000 appear to take 500 s). It is not clear what is necessary to apply the approach to higher-resolution parcellations.

Questions

1. What is the computational complexity of the algorithm? 2. Proposition 4 states that the initial permutation doesn't matter if some conditions are met. In practice, if these conditions are unmet or unknown, does the initial permutation matter?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

1: Your assessment is an educated guess. The submission is not in your area or the submission was difficult to understand. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

Limitations are discussed.

Reviewer 9NHG2023-08-14

I've read the rebuttal.

**The paper's focus on the approach also means little perspective is given until the final discussion.** In the discussion the assumptions and limitations are stated. In the discussion the family of graphs (ER and scale-free) underlying the DAG for the synthetic are mentioned. **The paper's claim of scaling** Although parcellations with more parcel's aren't common, couldn't arbitrarily small parcels be created down to the individual voxels through various techniques? My point is that it should be clarified how scalable the proposed techniques are, even if real data is not readily available. **What is the computational complexity** Thanks. **Proposition 4** The option to run it multiple times and the motivation (not assuming a DAG) should be clearly stated in the method section, not just the software, as should be the inclusion of fast causal inference as post-processing.

Reviewer 9NHG2023-08-14

confidence

I'll also add that my confidence is due to my own lack of in-depth familiarity with the causal discovery material. It is unfortunate that none of the reviewers are confident. I was added as an emergence reviewer and am not familiar with previous work: permutation-based scoring approaches, GRaSP, or GSTs. However, on reading some of the references I feel that this introduction is not ideal to those who aren't experts. I'm raising my score because I don't see problems, but I cannot assess the independently assess the correctness and true impact of this work without more background.

Authorsrebuttal2023-08-16

Thanks for the comments! Let me elaborate a bit...

Nice--I'm one of the other authors; the main author gave me permission to respond, haha, he will regret it... :-) Mentioning scale-freeness in the intro is a great idea. There is reasonable consensus among many now that fMRI connectivity is scale-free. GRaSP and BOSS deal with this sort of data very nicely, far better than the usual causal search algorithms, even out to an average degree of 20, without any compromise in quality. Also, we plan to rework the introduction in response to various comments. We're thinking maybe the "wow" factor of doing graphs with an average degree of 20 perhaps didn't come through well. Nearly all papers we've seen in Neurips and elsewhere deal only with very sparse graphs, average degree 2 or 4, or 6, which for a large number of variables is sparse in the extreme. We wanted to emphasize that our method could handle a much higher average degree without sacrificing accuracy (as almost all algorithms do, to the best of our knowledge) and responds well to many people's worries about doing causal searches accurately with dense graphs. DirectLiNGAM does reasonably well for dense graphs, but only under the stronger assumption of linear (strong) non-Gaussianity, and the empirical fMRI data we use are quite Gaussian, so it is not surprising (see Table 2) in our paper that DirectLiNGAM performs at chance levels for this sort of data. (Compare this to DAGMA in our charts, for instance, another Gaussian method published recently in Neurips and the best-performing continuous optimization algorithm we could find, so this is a comparison to that whole approach, so far as we currently know.) I was actually very excited to see the comment about scalability. For this paper, the state of the art for Gaussian searches for dense models is currently GRasP, though the published GRaSP scales only to about 100 variables, and in that paper, the maximum average degree tested was 10. In this paper, using a different permutation algorithm, we've increased the number of variables we can comfortably analyze 10-fold and the average degree we can analyze 2-fold without sacrificing accuracy. This sets a new state of the art for this problem, so far as we know. Notice also that even DirectLiNGAM, under a stronger assumption, takes much longer to run on this problem without appreciably better results; BOSS is able to match and even sometimes improve on the performance of DirectLiNGAM even for the linear, non-Gaussian case, which is helpful since even if you know that some variables are non-Gaussian you may not know that all variables are non-Gaussian. What excited me was that you wanted to extend the analysis to voxel-level analysis, which would to at least 40,000 variables. Genetic data also scales into that general dimension if one includes protein data in the analysis. We are keen to scale methods to that dimension for dense, scale-free data and are currently working through the various issues with parallelization, etc. However, there is no room in the current paper to address those optimizations; we are at the page limit. We plan to write another paper that does. It goes without saying that with good parallelization (not easy, key steps not embarrassingly parallel), we could jump from doing these analyses on laptops in a single thread to many-core machines. Also, we will definitely include the multiple-runs option in the methods section, thanks, and mention that for small numbers of variables (not so much for massive models), it can help. The idea of post-processing the BOSS output with a GFCI wrapper was proposed in the GRaSP paper but not explored, and we are hesitant to include any specific proposals on that before we've worked through the thorny issues of scaling a latent variable search up to the size problems that we are interested in. No one can do that now, even remotely, but we will try. We do have implementations of the procedure that people can try if they wish to see how well it does before such optimization. Using BOSS as an initial step certainly improves accuracy for these PAG methods. Regarding not being abreast of the permutation search literature, for one thing, GSTs are completely novel as of this paper, as is the BOSS algorithm, so we weren't expecting anyone to be familiar with those, no worries. The permutation methods, though, are very promising, as they currently outperform DAG (CPDAG) search over all other methods we know of for the linear Gaussian case by a long shot and match the best-performing algorithms for the linear non-Gaussian case. We're hoping to raise awareness of this methodology--i.e., hoping people will go back and read the Raskutti and Uhler paper, the Solus et al. paper, and the Lam et al. GRaSP paper and various other papers that have been published using permutations of variables orders as a key step.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC