FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective

Summary

The paper presents FourierGNN for multivariate time series forecasting from a pure graph perspective, performing matrix multiplications in Fourier space, which has not been investigated so far. They design a new hypervariate graph structure to consider spatiotemporal dynamics unitedly and reformulate the graph operations on the hypervariate graph in Fourier space. Extensive experiments have demonstrated superior performance with higher efficiency and fewer parameters over state-of-the-art methods.

Strengths

1. The paper is overall well-written. I think the contribution and novelty of the proposed work are clearly presented. 2. The idea of directly applying graph networks for multivariate time series forecasting seems novel and exciting. The hypervariate graph structure can encode spatiotemporal dependencies unitedly, and the visualizations in the experiments have also verified its advantages. 3. The authors argue that performing the multiplication between the input and the proposed FGO in Fourier space is equivalent to a graph convolution in the time domain, while the multiplications in Fourier space have lower complexity. The argument is proved by the authors and seems very meaningful. I think this argument provides a new path for conducting graph operations in Fourier space. 4. The effectiveness is validated through extensive experiments across seven real-world datasets. Sufficient analysis including efficiency analysis and ablation study is also provided.

Weaknesses

1. In the efficiency analysis section, the authors assess the overall efficiency of the forecasting process. To specifically evaluate the efficiency of FourierGNN, it would be more appropriate to focus on comparing the number of parameters involved in the graph operations with those of the baselines. 2. The MAPE (Mean Absolute Percentage Error) results on some datasets (e.g., Traffic, COVID-19) do not achieve state-of-the-art performance.

Questions

FourierGNN demonstrates impressive performance across multiple datasets, with notable success observed on the COVID dataset. Could you explain the reasons that contribute to the superior performance of FourierGNN on the COVID dataset compared to other datasets?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

N/A

Summary

This paper has studied a popular and important problem, i.e., modeling the intricate spatial and temporal dependencies among multivariate time series for accurate forecasting. To overcome the main limitation that existing works always separately model spatial and temporal, this work designs a new hypervariate graph structure to encode the spatiotemporal dynamics unitedly, and proposes a novel FourierGNN to learn the spatiotemporal dependencies. Extensive experiments on seven real-world datasets demonstrate FourierGNN achieves good performances in both accuracy and efficiency compared with state-of-the-art methods.

Strengths

This paper presents a novel formulation from a pure graph perspective to model spatiotemporal dependencies for MTS forecasting. The work is interesting and original, which is different from previous GNN-based methods that typically model spatial and temporal dependencies with distinct graph and temporal networks. The novel pure graph modeling is straightforward yet quite meaningful for MTS forecasting, which brings up good inspiring insights. This paper proposes a graph neural network, namely FourierGNN, to cooperate with the pure graph formulation and provides a theoretical guarantee of effectiveness. This paper seems to have solid, extensive and diverse experiments, which are conducted on seven real-world different datasets. The extensive experimental results have clearly demonstrated the obvious performance improvement of FourierGNN in terms of accuracy and efficiency over state-of-the-art MTS forecasting models.

Weaknesses

I have two concerns. My first concern is about the hypervariate graph structure. Although the authors stated that all variables at all timestamps can encode high-resolution relationships, it also may introduce some redundant information or unwanted correlations, such as some variables that are far apart in time. How can this be properly handled? Besides, I find that the core operations of FourierGNN are conducted in the Fourier space, so I am curious whether transferring the hypervariate graph structure into the Fourier space help to learn dependencies. In addition, another concern is whether FourierGNN can also be applied to other domains apart from the multivariate forecasting. Maybe it can be extended to other graph tasks. Minor typos: In the caption of Figure 2, Given the hypervariate graph $\mathcal{G}=(X^\mathcal{G}_t, A^\mathcal{G}_t)$ should be $\mathcal{G}_t=(X^\mathcal{G}_t, A^\mathcal{G}_t)$

Questions

1.Besides MTS forecasting, can FourierGNN be applied to other domains as well? 2.How does the proposed model handle the redundant information in the hypervariate graph structure?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

4 excellent

Limitations

n/a

Summary

The paper addresses the time series forecasting problem. The authors propose a model that represents each scalar observation as a node in a (fully connected) graph, encodes the nodes and finally regresses the future observations on all previous encoded observations (via a fully connected layer). As encoding they propose a discrete fourier transform followed by several node-wise fully connected layers and finally an inverse fourier transform. In experiments on 6 datasets they show improvements over several baselines.

Strengths

s1. interesting problem: time series forecasting s2. interesting approach: parametrizing transformations in fourier space s3. results showing improvements over many baselines

Weaknesses

w1. a main aspect of the proposed model is not clear. w2. missed recent related work. w3. experimental protocol is unclear and deviates from the literature. w4. for a key component of the proposed model, there is no ablation study demonstrating its impact.

Questions

w1. a main aspect of the proposed model is not clear. - the authors describe the fourier graph layer as applying the discrete fourier transform on the graph features X\in\R^{nd\times 1} of a complete graph, explicitly not the graph fourier transform. Which dimension is the time dimension for this fourier transform? The node dimension of X contains nd many entries, one for each time point and channel. While one can compute a fourier transform also over this vector, what is it semantics? How does it capture time varying information? And would this not crucially depend on how on is ordering these nd many entries? w2. missed recent related work. - esp. PatchTST [Nie et al. 2023] and dlinear [Zeng et al. 2023] are well known model outperforming your baselines by a good margin. w3. experimental protocol is unclear and deviates from the literature. - what is the forecasting horizon \tau you are using? - the experiments report only a single number per dataset and error measure, while in the literature usually results for a portfolio of forecasting horizons are reported (e.g., 96, 192, 336, 720, e.g., the Fedformer). Why do you deviate from this standard? This way it also is not possible to compare your results with the results published in those papers. - usually results are also reported for further datasets such as ETTm2, Exchange, Weather and ILI. How does your model compare to those published results? w4. for a key component of the proposed model, there is no ablation study demonstrating its impact. - while there are many papers that simply operate on the fourier spectrum of a time series, one key component of the model of the authors seems to be that after these operations they move back into the time domain by the inverse fourier transform. What is the impact of this back transformation? references: - Nie, Yuqi, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. “A Time Series Is Worth 64 Words: Long-Term Forecasting with Transformers.” arXiv, March 5, 2023. https://doi.org/10.48550/arXiv.2211.14730. - Zeng, Ailing, Muxi Chen, Lei Zhang, and Qiang Xu. “Are Transformers Effective for Time Series Forecasting?” In AAAI, 2023.

Rating

3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

n/a

Summary

In this paper, the authors study a problem in GNN-based multivariate time series (MTS) forecasting, i.e. modeling spatial correlations and temporal dependencies in the same time. In particular, the authors do not follow previous works on regarding the input as T graphs and capturing temporal dependencies between graphs by temporal networks, but propose to formulate T graphs as a hypervariate graph(a pure fully-connected graph with NT nodes). Specifically, to deal with the huge node amount NT in a hypervariate graph, the authors propose the FourierGNN which utilize matrix multiplications in the Fourier space of graphs to decrease the quadratic computational complexity to quasi linear one. Empirical results on seven datasets show the effectiveness of this method.

Strengths

1. The authors provide a novel view of graph-based MTS forecasting called hypervariate graph. 2. The authors propose Fourier Graph Operator and FourierGNN for efficient convolution on hypervariate graph. 3. In the authors’ experiments, FourierGNN obtains superior performance.

Weaknesses

1. The detail of the experiments is not clear. The authors run previous models on different datasets but do not show whether some hyperparameters, such as feature dim, change with the dataset. 2. The authors claim that FourierGNN is efficient for hypervariate graph, but do not provide other networks’ performance such as GCN on hypervariate graph. 3. The models compared with Efficiency Analysis are too old. 4. The authors do not explain the reason for the Green kernel. The Green kernel is a strong assumpition. In line 161, k_{ij} = A_{ij} W and k_{ij} = k_{i-j} results A_{j+k}_{j} equals for any j, which strictly restricts the topology of the hypervariate graph. Besides, there is no visualization of A in each layer to check whether the green kernel assumption is satisfied during training.

Questions

1. What’s the performance of FourierGNN on longer prediction lengths such as {24, 36, 48, 60} in FEDformer? 2. What’s the meaning of “the same sparsity pattern” in line 196? 3. How to determine the value of A_i in equation (7), A_0 is identity and what about A_1, A_2?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Please check the weakness

answer to rebuttal

Dear authors, thanks for your extensive answers and additional experiments. My concerns w1 and w4 you resolved. About the other two: w2. PatchTST and dlinear - The numbers you report for e.g. PatchTST deviate from the published numbers, e.g., for dataset weather, horizon 96, MAE - you report 0.034 (table 2 in your rebuttal pdf), - the PatchTST paper reports 0.198. Can you explain these differences? It would be more convincing to reproduce the experimental settings of the baseline papers (and their numbers). w3. experimental protocol unclear and deviates from the literature. 1. for the main experiment in tab. 1: what is \tau? 1 ? 2. what does "follow the experimental settings in short-term forecasting baselines, like LSTNet and StemGNN" mean exactly? - The LSTNet paper reports two different error measures, RSE and corr, so one cannot compare numbers directly. - The StemGNN paper reports RMSE for datasets Solar and Electricity, but they report different numbers than you do (their table 2): - Solar: they report 0.07, you report 0.222 - Electricity: they report 0.06, you report 0.101 Are you using a vastly different split? Or how are these differences explained? 3. Do any of the numbers in your table 1 coincide with some published numbers in the baseline papers? And if so, could you mark them, say with a star?