Deciphering Spatio-Temporal Graph Forecasting: A Causal Lens and Treatment

Summary

The paper introduces CaST, a novel framework designed to address challenges in Spatio-Temporal Graph (STG) forecasting. CaST tackles issues related to temporal out-of-distribution and dynamic spatial causation by leveraging a causal lens and employing techniques such as back-door adjustment and front-door adjustment. Experimental results on real-world datasets demonstrate the effectiveness and practicality of CaST, outperforming existing methods while providing good interpretability.

Strengths

Novel design of CaST. Thorough quantitative and qualitative analysis of the experiment results.

Weaknesses

There are too many design decisions that are made for CaST, making it hard to validate each individual component's contribution to the model performance. Missing complexity analysis.

Questions

Is the causal structure learned through a data driven approach or based on heuristic reasonings?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

I am debating if this is a limitation or not. Obviously the proposed method is composed of a lot of sophisticated design components. But maybe just considering causal structure in the time series data can be more compelling and easier to be generalizable.

Summary

The paper presents a novel framework, CaST, designed to address the challenges of temporal Out-Of-Distribution (OOD) issues and modeling the underlying dynamic spatial causation in Spatio-Temporal Graph (STG) forecasting. The authors construct a Structural Causal Model (SCM) to uncover the causal mechanisms of STG data, inspiring them to employ back-door and front-door adjustments to mitigate the confounding bias induced by temporal environment and spatial context, respectively. Moreover, they utilize the Hodge-Laplacian (HL) operator for edge-level convolution to capture the ripple effect of causation along space and time. Empirical results from experiments on three real-world datasets demonstrate that CaST outperforms existing methods while maintaining interpretability.

Strengths

1. The authors provide an in-depth analysis of the STG forecasting problem, revealing the underlying generation mechanisms of STG data and identifying the key sources of spatio-temporal distribution shift through the lens of causality. 2. The authors offer an interpretation of the specialties of temporal and spatial confounders (i.e., temporal environment and spatial contexts), making the application of backdoor and frontdoor adjustments in CaST targeted and well-motivated. 3. Extensive experiments validate the model's superiority, including comparisons against state-of-the-art STGNNs, ablation studies, hyperparameter sensitivity analysis, and case studies with detailed visualizations of the model's ability to capture spatial causality and identify temporal environments. 4. The figures provide readers with an intuitive understanding of the main problem, key ideas, and compelling experimental results.

Weaknesses

1. The concept of temporal disentanglement is not entirely novel. Although the specific designs of disentanglement headers for environment feature and entity feature may have some novelty, as shown in Figure 4(a), the authors did not adequately clarify this in the main content. 2. For the backdoor adjustment and corresponding temporal disentanglement module: a) The paper lacks a clear explanation regarding the connection between estimating $p(Y|do(X))$ via backdoor adjustment and temporal disentanglement, raising doubts about the effectiveness of the temporal disentanglement module. b) The proposed mutual information regularization cannot ensure that no causal features are leaked into the disentangled environment features $H_e$, possibly leading to the loss of essential causal information. 3. For the frontdoor adjustment and corresponding spatial context filtering module: a) The interpretation of the mediator variable $\hat{H}_i$ is confusing. The authors believe that $\hat{H}_i$ should be "a node representation containing only information propagated based on genuine causation within their spatial context". However, the spatial context $C$ is a confounder, and a mediator containing information from the confounder cannot satisfy the frontdoor criterion due to a backdoor path from the mediator to the label $Y,$ i.e., $\hat{H}_i \leftarrow C \rightarrow Y.$ b) The paper lacks a sufficient explanation regarding the connection between estimating $p(Y|do(X))$ via frontdoor adjustment and spatial context filtering, raising doubts about the effectiveness of the spatial context filtering. c) The reviewer fails to find an explanation for why using the HL operator on edge-graph can learn causation while other GNNs like GAT cannot. 4. The reference [1] addressed the spatio-temporal distribution shift in dynamic graphs. Since the spatio-temporal graph is a special type of dynamic graph, the authors should compare CaST with the approach described in [1]. 5. The ablation studies are insufficient. Validating the fact that **w/o Env** and **w/o Ent** can cause performance degradation is unnecessary, as it is a natural result of eliminating information predictive of the label. Additionally, the ablation studies on loss functions corresponding to the two disentanglers are missing. 6. The visualization of dynamic spatial causal relationships in section 5.2 is insufficient since the authors do not check whether other SOTA STGNNs can also capture these relationships. 7. The model's sensitivity to the hyperparameters $\alpha$ and $\beta$, which control the importance of different terms in the final loss function, should be discussed. [1] Zeyang Zhang, et al. Dynamic Graph Neural Networks Under Spatio-Temporal Distribution Shift. In NIPS 2022.

Questions

1. Why do the authors assume that temporal environment $E$ and spatial context $C$ are independent? For example, the states of neighboring nodes of node i will also be influenced by the change of $E$. 2. How do the proposed modules output statistical estimands that are required to calculate $p(Y|do(X))$ in Eqn. (1) and (2)? 3. Why do the authors focus on learning $p(Y|do(X))$ rather than $p(Y|do(X), E, C)?$ Although $E$ and $C$ are confounders, they can also provide additional information for accurate prediction. 4. In ablation studies, why **w/o Edge**, i.e., replacing 'GCN with causal scores' with GCN, causes such a performance drop in PEMS08? The STGCN also adopts GCN for spatial message passing, but it achieves MAE 18.60 in Table 1.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

see weakness

Summary

The paper introduces a model for out-of-distribution prediction in spatio-temporal data. The confounding effects are decoupled in spatial and temporal contexts, and treated with frontdoor and backdoor adjustments, respectively. Empirical evidence shows improved performance.

Strengths

- The tackled problem is extremely relevant. - To my knowledge, the proposed model consisting of a combination of edge-level filtering, back-door, and front-door adjustment is novel and sound. - The achieved performance is remarkable.

Weaknesses

- It's unclear to me to what extent the considered assumption of decoupled spatial and temporal environments is reasonable. For instance, the weather (mentioned in line 124) is related to both time and space. - The experimental setup is not ideal in my opinion.

Questions

### Questions: - Line 282: Most of the literature considered a different number of steps (see, eg, DCRNN and MTGNN). Although different choices are valid as well, considering the same setting provides a means to assess reproducibility and allows a broader comparison of the methods. Is there a particular reason for not considering these settings? ### Suggestions: - The top and bottom halves of Fig. 3 referred to in Sec. 4 are not easily identified in the figure. I suggest highlighting them, e.g., with boxes or shaded areas. - Line 211: This is guaranteed only if the KL divergence is 0. I suggest rephrasing it. - The causal strength constitutes an important element. I suggest providing a proper definition and discussion about it. - Other methods in the literature are generally winning over the methods considered in the paper (see e.g. [https://arxiv.org/abs/2005.11650](https://arxiv.org/abs/2005.11650) and the reference therein). I suggest considering some of them, like Graph WaveNet [48] which is also a quite established baseline from 2019. ### Typos - Line 162: Eq. 8. ### After rebuttal I have read the author's rebuttal which has addressed all the raised points.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

yes

Summary

This paper studies the problem of Spatio-Temporal Graph forecasting under the lens of causal treatments. The authors proposed a framework consisting of two major components to mitigate the commonly seen limitations for spatial-temporal GNNs, i.e., 1) the backdoor environment disentanglement block, which models the temporal environmental changes, and 2) the Hodge Laplacian deconfounder capturing the spatial context. The proposed framework is applied in the air and traffic flow datasets to study spatial-temporal interactions.

Strengths

1. [Originality/Significance] The framework is proposed to deal with the limitation of the current STGNN models, i.e., the out of distribution issue and dynamic spatial causation. This paper’s originality comes from a novel combination of causal back-door treatment for temporal components, and a Hodge Laplacian decoupling layer for the spatial contexts. 1. [Quality] The overall experiments presented in this paper for supporting the proposed CaST pipeline are quite nice. It contains different aspects such as ablation study on the core components of the model, which can provide deep insight into the model. 1. [Clarity] The paper is well-written, with detailed descriptions starting from the background, STG data generation causal graph, causal treatment formulation, to each building block in the CaST pipeline.

Weaknesses

1. From Appendix D (L646), it looks like the Hodge Laplacian used here is the down Hodge Laplacian $\partial_1^\top\partial_1$, rather than the full Laplacian (because $\partial_2$ is set to zero here). I would suggest mentioning this in the main text, because this is a big assumption to make. Additionally, there is some existing work using Hodge Laplacian on graphs (e.g., [A]), I would suggest citing this paper for completeness. 1. Related to #1, after applying the edge filter you created, the filtered edge signal will be divergence free [B,C] (due to the low frequency/null space of the down Laplacian being the curl or harmonic flows). This implies that any gradient edge signal (i.e., a flow $x\in\mathbb R^{|E|}$ that can be expressed as $x = \partial_1 y$ for some node function $y \in \mathbb R^{|V|}$) will very likely be filtered out after the convolutional layer, suggesting that the HL deconfounder can learn more information when the flow is divergence free (compared with a curl-free flow). Given that the traffic flow is incompressible (thus divergence free) while air (PM2.5) flow is not, it might also suggest why we see huge drop in performance for the ablation study (Figure 5a) when removing edge signal on the PEM508 (traffic flow) dataset, compared with the AIR_BJ (air flow). I would suggest discussing this assumption/limitation in more detail in the main text. --- [A] Schaub, Michael T., Austin R. Benson, Paul Horn, Gabor Lippner, and Ali Jadbabaie. “Random Walks on Simplicial Complexes and the Normalized Hodge 1-Laplacian.” SIAM Review 62, no. 2 (2020): 353–91. [B] Chen, Yu-Chia, Marina Meilă, and Ioannis G. Kevrekidis. “Helmholtzian Eigenmap: Topological Feature Discovery & Edge Flow Learning from Point Cloud Data.” ArXiv:2103.07626 [Stat.ML], March 13, 2021. https://arxiv.org/abs/2103.07626v1. [C] Schaub, M. T., and S. Segarra. “Flow Smoothing And Denoising: Graph Signal Processing In The Edge-Space.” In 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP), 735–39, 2018. https://doi.org/10.1109/GlobalSIP.2018.8646701.

Questions

1. Related to the weakness #1, have you tried/considered constructing a $\partial_2$ from e.g., a clique-complex (filling all triangles in the edges), and see if there is performance gain by using the full up and down Hodge Laplacian? 1. In L275, is there any specific reason to regularize only the scaling $\beta$ of $\mathcal L_{mi}$ rather than $\mathcal L_{cod}$? I understand that there is a regularization parameter $\alpha$ in Eq. (6), but how do you make sure the first term of $\mathcal L_{cod}$ is comparable with the log-probability ($\mathcal L_{pre}$)? 1. In L745-750 of Section G, the authors mentioned the edge filter being computationally intensive. If there is space, I would like to see an empirical comparison on the runtime. 1. How should I correctly understand the removal of the Environment codebook in the ablation study (Figure 5a)? Specifically, it seems like the gain for adding Env feature is not huge, does it mean that the environment disentangler is not disentangle the entity from environment well enough (so that some “environmental features” are still presented in the entity features)?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors have discussed extensively the limitations as well as the social impact in the Appendix, therefore, I do not have any other aspects to add. While I appreciate the detailed discussions in Section G, I believe that it will improve the paper even more if the authors can at least have a brief overview (3-5 sentences) of the limitation/social impact in the main text.