Summary
The paper presents a Spectral Attention mechanism to address the challenge of capturing long-range dependencies in time series forecasting. By preserving temporal correlations among samples and leveraging frequency domain information, the proposed method enhances the forecasting performance of various baseline models. Extensive experiments demonstrate the efficacy of Spectral Attention, achieving state-of-the-art results on multiple real-world datasets.
Strengths
1. Efficacy: The proposed Spectral Attention mechanism significantly improves forecasting performance across multiple real-world datasets.
2. Versatility: The method can be integrated into various baseline models, showcasing its adaptability and broad applicability.
3. Experimental validation: The experiments show consistent performance improvements.
Weaknesses
1. Novelty and Contribution/Missing Related Work
* While the paper introduces Spectral Attention as a novel method, using frequency domain analysis in time series forecasting is not a new concept. Notably, there is an existing paper [1] with the same title, "Spectral Attention," also focused on time series forecasting, which is not referenced in the manuscript. This referenced paper employs a global/local perspective: the global Spectral Attention provides comprehensive information about the random process the time series are considered part of, thereby addressing some limitations mentioned in the paper, such as the inability to model long-range dependencies. Furthermore, the referenced work includes spectral filtering, differing from the authors' approach by learning a cut-off frequency while filtering each frequency component independently. Their solution integrates easily into pre-existing architectures. Given the significant similarities and the relevance of this other paper as a precursor to the authors' idea, why is it not cited in the manuscript? The authors must include a thorough discussion of [1], acknowledging its relation and relevance to their work.
* Additionally, recent SSM-based approaches should be covered in the related work section.
2. Differentiation from Existing Methods
* The paper would benefit from a clearer distinction between Spectral Attention and existing frequency-based methods such as WaveFroM, FEDformer, or the Spectral Attention Autoregressive Model. The unique advantages and innovations of Spectral Attention compared to these methods should be emphasized more explicitly.
3. Computational Complexity
* The paper claims that Spectral Attention adds minimal computational overhead. However, the analysis of computational complexity and memory requirements is somewhat superficial. A detailed comparison of training and inference times, as well as memory usage, with and without Spectral Attention, should be provided to substantiate these claims.
4. Integration with Different Models
* While the paper demonstrates that Spectral Attention can be integrated with various TSF models, the integration process lacks detail. Specific guidelines or algorithms for integrating Spectral Attention with different architectures should be included. This would facilitate practitioners in applying the proposed method to their models.
[1] Moreno-Pino, Fernando, Pablo M. Olmos, and Antonio Artés-Rodríguez. "Deep autoregressive models with spectral attention." Pattern Recognition 133 (2023): 109014.
Questions
1. How does the proposed method specifically improve upon the referenced Spectral Attention Autoregressive Model [1]?
* A detailed comparative analysis and, if possible, empirical validation are necessary to substantiate the claims of improvement.
2. Can you provide a more detailed explanation of the complexity involved in the spectral attention mechanism?
3. Given that the proposed model operates in the frequency domain, is it capable of handling irregularly sampled data even if the underlying architecture cannot?
I am open to raising my score if the authors adequately address these concerns.