Are Self-Attentions Effective for Time Series Forecasting?

Time series forecasting is crucial for applications across multiple domains and various scenarios. Although Transformer models have dramatically advanced the landscape of forecasting, their effectiveness remains debated. Recent findings have indicated that simpler linear models might outperform complex Transformer-based approaches, highlighting the potential for more streamlined architectures. In this paper, we shift the focus from evaluating the overall Transformer architecture to specifically examining the effectiveness of self-attention for time series forecasting. To this end, we introduce a new architecture, Cross-Attention-only Time Series transformer (CATS), that rethinks the traditional Transformer framework by eliminating self-attention and leveraging cross-attention mechanisms instead. By establishing future horizon-dependent parameters as queries and enhanced parameter sharing, our model not only improves long-term forecasting accuracy but also reduces the number of parameters and memory usage. Extensive experiment across various datasets demonstrates that our model achieves superior performance with the lowest mean squared error and uses fewer parameters compared to existing models. The implementation of our model is available at: https://github.com/dongbeank/CATS.

Paper

References (32)

Scroll for more · 20 remaining

Similar papers

Peer review

Reviewer ztsM6/10 · confidence 4/52024-06-19

Summary

This work proposed a cross-attention only transformer model for time series forecasting

Strengths

Interesting work. This study thoroughly examines the effectiveness of cross-attention layers in transformers and proposes several useful techniques to build a high-performance model.

Weaknesses

1. It is intriguing to explore why self-attention is not beneficial for time series forecasting. While the experimental results are robust, readers are more interested in understanding the underlying reasons. 2.The proposed query-adaptive masking contributes to performance improvement but also weakens the overall claim. The added complexity raises questions about whether cross-attention is truly the key factor or if similar designs could enhance self-attention as well. This relates back to the 'why' issue—what is really happening with self-attention and cross-attention in time series forecasting, and why is one mechanism potentially superior to the other?

Questions

In the weakness.

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

Not applicable

Authorsrebuttal2024-08-07

Rebuttal by Authors (Cont')

[W2 - Is the model structure inherently efficient, or is self-attention inherently inefficient?] We would like to respond with "Both." Let us denote the input length as $L$ and the target length as $T$. The complexity of self-attention grows quadratically with the input length, $O(L^2)$, as each element attends to every other element in the sequence. Additionally, it requires an extra linear layer to interpret the attention representation with respect to the target length $T$. This leads to high computational resource demands as the sequence length increases. In contrast, cross-attention only requires $O(LT)$ complexity. The linear complexity of cross-attention makes it inherently more scalable and efficient for long sequences. Table 4 explicitly illustrates this phenomenon. In summary, self-attention-based transformers are inherently inefficient as $L$ increases compared to our model. Unfortunately, the proposed techniques such as 'parameter sharing across horizons' and 'query-adaptive masking,' cannot be directly applied to existing models with self-attention. To ease understanding, we summarize the applicability of our techniques for different model types in the table below. | | Parameter Sharing across Horizons | Query-adaptive Masking | | --- | --- | --- | | Transformer-based Models (e.g., PatchTST, Crossformer, …) | ❌ Not applicable due to concatenation in the final fully-connected layer. | 🔺 Applicable but affects all horizons rather than the related horizon. | | Linear-based Models (e.g., TimeMixer, DLinear, …) | ❌ Not applicable as forecasting horizons are not separated. | ❌ Not applicable as these models do not use queries. | | CATS (proposed) | ✔️ Applicable as each horizon is independently calculated. | ✔️ Applicable and effective as masking affects only the related horizon. | While transformer-based models can adopt query-adaptive masking, it affects all horizons rather than the horizon related to the masked query. This is expected to decrease model performance, and indeed, we observe the expected results as follows: | | PatchTST | PatchTST + Query-adaptive Masking | Ours | | --- | --- | --- | --- | | Average MSE | 0.353 | 0.409 | 0.338 | | Average MAE | 0.382 | 0.417 | 0.376 | In other words, adopting these techniques is not available to existing methods; however, as shown in Table 5, we believe that adopting cross-attention in existing methods can alleviate computational costs and improve performance, which remains a direction for future work. Thank you again for your kind feedback.

Reviewer ztsM2024-08-08

Some of my questions have been addressed, and the paper is quite intriguing. I will increase the score accordingly.

Authorsrebuttal2024-08-09

Thank you for your feedback

We are very pleased to hear that our response has addressed your concerns. We truly believe that the quality of the paper has significantly improved thanks to the valuable comments provided by the reviewer. Once again, we would like to express our sincere appreciation for your decision to increase the score, and thank you for your time and expertise.

Reviewer Q7Js3/10 · confidence 5/52024-07-13

Summary

The paper titled "Are Self-Attentions Effective for Time Series Forecasting?" introduces a novel time series forecasting architecture named Cross-Attention-only Time Series transformer (CATS). The central hypothesis of the paper is that self-attention mechanisms, a key component of the Transformer models, may not be as effective for time series forecasting as simpler linear models. The CATS model enhances parameter sharing and improves long-term forecasting performance by using future horizon-dependent parameters as queries and past time series data as key and value pairs. The authors conducted extensive experiments across various datasets and demonstrated that CATS achieves superior performance with the lowest mean squared error and uses fewer parameters compared to existing models.

Strengths

The paper proposes a novel forecasting model, CATS, which innovatively removes self-attention mechanisms in favor of cross-attention, offering a fresh perspective on time series forecasting architectures. The model requires fewer parameters and less memory consumption compared to existing Transformer models, which enhances its efficiency, especially beneficial for large-scale applications. By eliminating self-attention, the proposed model simplifies the overall structure, which could lead to better interpretability and understanding of the forecasting process.

Weaknesses

The weaknesses of the paper "Are Self-Attentions Effective for Time Series Forecasting?" from the perspectives provided can be summarized as follows: 1. The use of an embedding-based cross-attention method is not a novel approach in the field of time series forecasting. The paper's reliance on this method may not offer a significant advancement over existing techniques. 2. The paper's expression is difficult to read, which hinders the understanding of its contributions. Sections like 3.2, which discusses 'cross-attention via future as query,' are particularly hard to comprehend, suggesting that the ideas could be presented more clearly. 3. The concepts of 'parameter sharing across horizons' and 'query-adaptive masking' appear to be quite standard design choices in the context of neural network architectures. The paper might be overstating their significance as core novel structures when, in reality, these could be considered trivial or expected features in modern models. 4. If the aforementioned design elements are indeed the core modules of the proposed CATS architecture, their presentation as innovative contributions might be unconvincing. The paper needs to provide stronger justification for considering these elements as the foundation of its novelty. 5. The paper's central claim revolves around the effectiveness of self-attentions for time series forecasting, but the proposed solution does not introduce a groundbreaking concept. Instead, it modifies existing structures, which might not be enough to establish a new baseline in the field.

Questions

See weaknesses.

Rating

3

Confidence

5

Soundness

2

Presentation

1

Contribution

2

Limitations

See weaknesses.

Area Chair LeRM2024-08-12

Discussion

Thank you for being a reviewer for NeurIPS2024, your service is invaluable to the community! The authors have submitted their feedback. Could you check the rebuttal and other reviewers' comments and start a discussion with the authors and other reviewers? Regards, Your AC

Reviewer Q7Js2024-08-13

The backbone structure of this paper is very similar to PatchTST, with the core difference being the introduction of the cross-attention module and the use of the shared projection layer mentioned in line 171 (which was claimed to be better designed separately in the PatchTST paper). Therefore, while I agree with the authors on the importance of parameter sharing, the supplementary experiments here bring greater confusion. Specifically, it is quite unusual that PatchTST performs significantly worse than CATS after introducing Query-adaptive Masking. This anomaly requires additional experimental details, and the paper needs to submit an updated version with revised experiments and re-emphasized importance. Furthermore, I think the introduction of the cross-attention mechanism and parameter sharing mechanism makes sense in that this method limits the model size, thereby achieving model gains. Many papers have discussed the generalization issues caused by the introduction of attention or excessive parameters in PatchTST, to the extent that linear models can perform better than PatchTST on some tasks. However, the authors mainly emphasize cross-attention itself in the paper. Without supplementary experiments or a reasonable explanation in this regard, the paper risks being significantly misleading. In conclusion, due to substantial concerns about the paper, I will maintain my score for the current manuscript.

Authorsrebuttal2024-08-13

Rebuttal by Authors

Thank the reviewer for the response. **First of all, we strongly argue that the backbone structure itself is our novelty, which is NOT similar to PatchTST.** We would like to highlight that all other reviewers have agreed with our argument as follows: *"The introduction of **CATS represents an innovative approach in the field of time series forecasting**, particularly in re-evaluating the role of self-attention mechanisms. (by **Reviewer sgNU**)"*. Especially, **'the core difference being the introduction of the cross-attention module' itself is also our main contribution,** which has not been covered in time series domain. *"Interesting work. This study thoroughly examines **the effectiveness of cross-attention layers** in transformers and proposes several useful techniques to build a high-performance model. (by **Reviewer ztsM**)"* Therefore, **we respectfully disagree with the assessment that "the backbone structure of this paper is very similar to PatchTST."** While we acknowledge that PatchTST is a foundation model in time series domain, CATS is completely different from PatchTST due to our cross-attention-based backbone and the corresponding data-flow. --- *Further details addressing this and other comments are provided in the following sections.*

Authorsrebuttal2024-08-13

Rebuttal by Authors (Cont')

Based on your comments, we have identified and categorized the weaknesses [W#] you highlighted into three key areas: Structural Similarity with PatchTST, Query-Adaptive Masking in PatchTST and CATS, and Lack of Emphasis on Broader Comparisons. --- **[W1 - Structural Similarity with PatchTST]** **Comment:** *"The backbone structure of this paper is very similar to PatchTST, with the core difference being the introduction of the cross-attention module and the use of the shared projection layer."* While both CATS and PatchTST utilize patching, it’s important to clarify that beyond this similarity, the two models diverge significantly in their overall architecture and approach to time series forecasting. PatchTST emphasizes univariate forecasting and patching while largely retaining the standard Transformer structure by keeping the vanilla Transformer encoder intact and simply substituting the decoder with a linear layer. In contrast, CATS takes a more radical departure from traditional Transformer models by **not only eliminating the self-attention mechanism but also redesigning the decoder**. Unlike PatchTST, which maintains the vanilla Transformer encoder, CATS removes the masked self-attention from the decoder as well, effectively addressing the challenges that arise from this change. Eliminating self-attention introduces significant challenges, particularly in time series forecasting. These challenges can be summarized as follows: 1. **Inability to Generate Necessary Queries** - **Problem:** Without self-attention, the fixed input tokens used in traditional decoders are insufficient to generate the necessary queries for cross-attention. - **Solution:** We introduced learnable queries, establishing horizon-dependent parameters as learnable queries. This ensures that each forecasting query independently generates the necessary input for cross-attention. 2. **Temporal Information Loss Due to Concatenated Linear Layers** - **Problem:** Concatenated linear layers at the end of the decoder could lead to temporal information loss. - **Solution:** We designed the decoder to process each horizon's query independently using parameter sharing across layers. This approach minimizes temporal information loss by ensuring that each query's processing remains independent, thus preserving temporal information. Given that PatchTST closely aligns with the vanilla Transformer structure—using an identical encoder and only replacing the decoder with a linear layer—such reasoning could be extended to suggest that any model based on the vanilla Transformer architecture is "very similar" to PatchTST. However, **the introduction of novel mechanisms in CATS results in a distinct decoder architecture from traditional Transformers, designed to be more efficient and effective for time series forecasting, setting CATS apart from both the vanilla Transformer and PatchTST.** --- **[W2 - Query-Adaptive Masking in PatchTST and CATS]** **Comment:** *"It is quite unusual that PatchTST performs significantly worse than CATS after introducing Query-Adaptive Masking."* Regarding Query-Adaptive Masking, as mentioned in our rebuttal, applying this technique to PatchTST leads to significant performance degradation because masking a single query impacts the entire output. However, as detailed in Appendix B.3, CATS is designed to handle "**multiple inputs with different forecasting horizons sharing a single model** (in line 493)". This structure allows for the masking of individual queries, which only affects the specific horizon they pertain to. This probabilistic masking enables the model to focus more effectively on learning from the unmasked queries, thereby avoiding the performance pitfalls observed in PatchTST.

Authorsrebuttal2024-08-13

Rebuttal by Authors (Cont')

**[W3 - Lack of Emphasis on Broader Comparisons]** **Comment:** *"The authors mainly emphasize cross-attention itself in the paper. Without supplementary experiments or a reasonable explanation in this regard, the paper risks being significantly misleading."* We strongly disagree with the assertion that this paper "mainly emphasizes cross-attention itself in the paper without supplementary experiments." We fully acknowledge the recent research highlighting the parameter and memory efficiency challenges of PatchTST and the resulting advantages of linear models. Therefore, **our study includes a comprehensive comparison not only with PatchTST but also with TimeMixer, a state-of-the-art linear model.** In Section 4.2, **we explicitly address the parameter and memory efficiency issues by comparing our model with both linear and Transformer-based state-of-the-art models.** The results of these comparisons are detailed in Table 4 of our paper, which clearly illustrates the advantages of CATS over both PatchTST and linear models. * Specifically, we observed that PatchTST suffers from significant memory complexity issues. As the input length increases from 336 to 2880, PatchTST's memory usage escalates by 16.9 times. Linear models were developed to address such issues, and indeed, they reduce the memory usage increase to 3.6 times over the same input length extension. However, our CATS model performs even better, with only a 1.4 times increase in memory usage. * On the parameter side, linear models exhibit a drastic increase of 46.8 times in parameters as the input length increases, while CATS shows a minimal increase of only 1.1 times. Even the theoretically simplest model, DLinear, shows an 8.5 times increase in parameters, far exceeding the minimal increase observed in CATS. These results demonstrate that **our study does not simply focus on cross-attention or comparison with PatchTST; rather, it comprehensively evaluates our model against linear models as well**, showcasing the superior efficiency of CATS in both memory and parameter usage. In addition, the additional results of these experiments are provided in **Appendix B.2: Additional Results for Section 4.2**, which thoroughly documents our findings. In summary, **we belive that we provide a broad comparison across the landscape of time series forecasting models**, thereby proving the superiority of CATS. **Please let us know in more detail what experiments the reviewer is curious about** for _"This anomaly requires additional experimental details."_

Reviewer sgNU6/10 · confidence 3/52024-07-14

Summary

The paper presents a novel architecture, the Cross-Attention-only Time Series transformer (CATS), which challenges the conventional use of self-attention mechanisms in Transformers for time series forecasting. The authors propose a model that leverages cross-attention instead, aiming to enhance long-term forecasting accuracy while reducing parameter count and memory usage.

Strengths

The introduction of CATS represents an innovative approach in the field of time series forecasting, particularly in re-evaluating the role of self-attention mechanisms. The paper highlights significant improvements in computational efficiency through reduced parameter sharing and memory usage.

Weaknesses

1. in the paper and the supplementary code, the training epoch is set to 30 and the early stop patience is set to 10, which is inconsistent with the baseline settings used for comparison, raising concerns about unfair comparisons. 2. The main experiments in the article are conducted only on long-term time series forecasting, without including short-term time series forecasting on datasets such as M4.

Questions

as above

Rating

6

Confidence

3

Soundness

3

Presentation

2

Contribution

2

Limitations

None

Area Chair LeRM2024-08-12

Discussion

Thank you for being a reviewer for NeurIPS2024, your service is invaluable to the community! The authors have submitted their feedback. Could you check the rebuttal and other reviewers' comments and start a discussion with the authors and other reviewers? Regards, Your AC

Reviewer sgNU2024-08-12

Thank you for your explanation. I believe the contribution of this article is roughly enough, but the writing does make it somewhat difficult to quickly grasp its contribution. I will raise my score to 6.

Authorsrebuttal2024-08-13

Thank you for your detailed feedback. We greatly appreciate your insights, which have not only helped us to make our results more robust, but also allowed us to significantly improve the clarity and quality of our writing. We are pleased to hear that you believe the contribution of this article is sufficient, and we sincerely appreciate your decision to raise the score. Your time and expertise have been invaluable in strengthening our work.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC