Ada-MSHyper: Adaptive Multi-Scale Hypergraph Transformer for Time Series Forecasting

Summary

This paper presents an adaptive hypergraph learning module for modeling group-wise multi-scale interactions to improve transformer-based model for time series data. Given a time series data, Multi-Scale Feature Extraction (MFE) Module first converts it to a hypergraph. Then, intra-scale and inter-scale learning modules are performed. Comprehensive experiments demonstrate the effectiveness of the proposed method.

Strengths

1. The motivation behind Ada-MSHyper is well-founded and insightful. 2. The design of the proposed method effectively addresses the target challenge. 3. Extensive experiments were carried out, demonstrating the clear superiority of Ada-MSHyper. 4. The writing of the paper is clear and easy to follow. Overall, this is a solid paper with excellent presentation.

Weaknesses

This paper is generally well-written. My only suggestion is to provide more insights into the performance differences of Ada-MSHyper across different datasets. Consider analyzing the reasons behind its performance on long-range, short-range, and ultra-long-range datasets. Does Ada-MSHyper perform better on one type compared to the others? What might be the reasons for this? For example, the node constraints on different dataset could be quite different. Node clustering might behave poorly on datasets with weaker temporal patterns. Answering these questions can help understand the effectiveness and limitations of Ada-MSHyper.

Questions

1. Considering that one of the challenges addressed by this paper is "temporal variations entanglement," how do the authors compare their method with frequency domain analysis techniques? For instance, have they considered converting the time-series data into a spectrogram using Short-Time Fourier Transform? Can Ada-MSHyper easily adapt to the 2D spectrogram data in time-frequency domain?

Rating

7

Confidence

3

Soundness

4

Presentation

4

Contribution

3

Limitations

More insightful analysis could be added to the experiments. See weaknesses and questions.

Summary

(1) Design an AHL module to model the abundant and implicit group-wise node interactions and a multi-scale interaction module to model group-wise pattern interactions at different scales. (2)Introduce a NHC mechanism to cluster nodes with similar semantic information and differentiate the temporal variations within each scales.

Strengths

(1) The first work that incorporates adaptive hypergraph modeling into time series forecasting; (2) Design AHL module to solve semantic information sparsity, and NHC mechanism to solve temporal variations entanglement, it is interesting; (3) Achieve state-of-the-art (SOTA) performance. (4) Lots of experiments and ablation studies, and visualizations to demonstrate the effectiveness of the proposed methodology.

Weaknesses

(1) The pipeline is kind of complicated, and the color of it can be improved; (2) Some expression in the paper can be more formal, like “entries hnm” can be replace by “entries Hnm” for more formal.

Questions

(1) In your Hyperedge Constraint, you use both cosine similarity α and Euclidean distance D, are your Euclidean distance normalized? It seems that cosine similarity and normalized Euclidean distance are similar, so will it provide too much redundant information to use αD as your hyperedge loss? Can you do some ablation studies? (2) I am kind of confused about your Multi-Scale Interaction Module, why your Intra-Scale Interaction part use HGNN for message passing, and your Inter-Scale Interaction Module part use transformer for updating features, is there any reason for this? Cause both interaction part are using attention, can they just use the same method for updating features? (3) Note that you use lots of attention mechanism to model group-wise pattern interactions, you use the cosine similarity and the Euclidean distance, both are known to us well. I wonder whether there are some other distance for modeling the pattern interactions we can try? Like Wasserstein distance.

Rating

4

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Authors addressed the limitations in appendix I that their datasets are not really large, so the generalization capabilities of their models may not be really good. The paper mainly focuses on scientific research and has no obvious negative social impact.

Summary

This paper presents a Time Series Forecasting method Ada-MSHyper that uses a hyper graph to capture the group-wise interactions at different time scales rather than Point-Wise interaction. Experiments are performed on 8 data sets and the proposed method is compared with SOTA methods

Strengths

1. The use of Hyper Graph 2. Differentiation of variation at Each Time Scale 3. Comparisons with SOTA Methods 4. Ablation Study 5. Comparisons of Computational Cost with 3 SOTA

Weaknesses

1. Related works does not include Graph-Transformer methods e.g. STGNN 2. Graph-Transformer methods e.g. STGNN are not used as SOTA for comparisons. 3. Data Sets should include financial data e.g. Stock Market 4. In results of "Ultra-Long-Range Forecasting", the reason for comparebale accuracies of "WITRAN" on ETTm2 5. The ablation study should include the effect of "η is the threshold of T op K function"

Questions

1. Why SOTA methods do not include Graph-Transformer methods e.g. STGNN? 2. Will this process of "reduce subsequent computational costs and noise interference", introduce any loss of usefull information? 3. Have you studied and quantified the computational cost reduction by the above approach? 4. Is the linear layer for forecasting a linear regression? 5. Will the proposed method work on Stock Market Data? 6. Have you analyzed the data sets to check and quantified the presence of "multi-scale pattern interactions"?

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

No Applicable

Summary

This paper introduces a hypergraph-based multi-scale time series forecasting model. By treating multi-scale feature representations as nodes, the proposed AHL module automatically generates incidence matrices to model implicit group-wise node interactions at different scales. Node constraints and hyperedge constraints are introduced to effectively aggregate similar semantic nodes and distinguish temporal variations at different scales. The experimental results confirm the predictive ability of the model, and it can still maintain low MSE and MAE on the ultra-long range series.

Strengths

1. The authors propose the AHL module, which is applied to a hypergraph constructed from time series, enabling the discovery of pattern interactions at various scales. 2. The experimental results demonstrate that the model has achieved state-of-the-art (SOTA) performance in short, long, and ultra-long sequence prediction. 3. The paper exhibits a clear structure and substantial content. It provides a detailed introduction to the structure and function of each module in the model.

Weaknesses

1. In the ablation experiment, only ETTh1 dataset was utilized to conduct experiments at three prediction lengths: {96, 326, 720}. This setup lacks support for evaluating ultra-long-range performance. For instance, in the case of w/o NC (without Noise Conditioning), the performance difference between the proposed model and its counterpart becomes only marginally noticeable as the sequence length increases. Since there is a lack of additional datasets and ultra-long-range experiments, it is reasonable to interpret this result as a normal experimental error rather than a significant performance difference. 2. The experiments conducted to assess model efficiency are too simplistic to effectively validate the author's viewpoint.

Questions

1. The paper claims that most of the experimental results for long-range prediction come from the DLinear model. However, there are no experimental results of the Crossformer model in the DLinear paper. Moreover, the results presented in this paper differ significantly from those in the ITransformer paper for certain datasets. It would be helpful to have an explanation for these differences. 2. In Section 4.1, the paper introduces multi-scale sequence construction, while Section 4.2 discusses the AHL module. One aspect that raises curiosity is the process of mapping from the sequence to the hypergraph. Specifically, is each time point with multiple features in the time series mapped to a node in the hypergraph? 3. Regarding model efficiency, the paper provides a comparison of model parameters and training time on a specific dataset. However, a single experiment result may not be sufficient to establish convincing evidence. It would be beneficial to explain the model's efficiency from the perspective of theoretical complexity or provide information on model parameters and training time for longer input sequences and on additional datasets. 4. In the ablation experiment section, it would be beneficial to include experiments with longer prediction lengths to further evaluate the model's performance.

Rating

7

Confidence

4

Soundness

3

Presentation

4

Contribution

3

Limitations

The authors adequately addressed the limitations