FilterNet: Harnessing Frequency Filters for Time Series Forecasting

While numerous forecasters have been proposed using different network architectures, the Transformer-based models have state-of-the-art performance in time series forecasting. However, forecasters based on Transformers are still suffering from vulnerability to high-frequency signals, efficiency in computation, and bottleneck in full-spectrum utilization, which essentially are the cornerstones for accurately predicting time series with thousands of points. In this paper, we explore a novel perspective of enlightening signal processing for deep time series forecasting. Inspired by the filtering process, we introduce one simple yet effective network, namely FilterNet, built upon our proposed learnable frequency filters to extract key informative temporal patterns by selectively passing or attenuating certain components of time series signals. Concretely, we propose two kinds of learnable filters in the FilterNet: (i) Plain shaping filter, that adopts a universal frequency kernel for signal filtering and temporal modeling; (ii) Contextual shaping filter, that utilizes filtered frequencies examined in terms of its compatibility with input signals for dependency learning. Equipped with the two filters, FilterNet can approximately surrogate the linear and attention mappings widely adopted in time series literature, while enjoying superb abilities in handling high-frequency noises and utilizing the whole frequency spectrum that is beneficial for forecasting. Finally, we conduct extensive experiments on eight time series forecasting benchmarks, and experimental results have demonstrated our superior performance in terms of both effectiveness and efficiency compared with state-of-the-art methods. Code is available at this repository: https://github.com/aikunyi/FilterNet

Paper

Similar papers

Peer review

Reviewer t3Wx7/10 · confidence 4/52024-07-08

Summary

This paper introduces FilterNet, a novel neural network architecture for time series forecasting that leverages learnable frequency filters inspired by signal processing techniques. This paper proposes two types of filters within FilterNet: a plain shaping filter that uses a universal frequency kernel for signal filtering and temporal modeling, and a contextual shaping filter that examines filtered frequencies for compatibility with input signals to learn dependencies. This paper demonstrates that FilterNet can effectively handle high-frequency noise and utilize the full frequency spectrum, addressing key limitations of existing Transformer-based forecasters. This paper shows through extensive experiments on eight benchmarks that FilterNet achieves superior performance in both effectiveness and efficiency compared to state-of-the-art methods in time series forecasting.

Strengths

1. This paper studies an interesting research direction, which is to adapt the advanced techniques in the signal processing domain into the time series forecasting, rather than following the existing transformer-based time series works. This is the original and novel contribution to the time series community. 2. The main model part is overall descent. The simple but effective architectures built upon the frequency filters are proposed. I notice there is not many redundant components in the basic model, compared with existing transformer models. It can somehow demonstrates the effectiveness of their proposed two filters. 3. The experiments are solid and convincing. Several experimental results are reported on existing benchmark datasets. Also, some experiments on synthetic periodic and trend signals with noises shown in experiments make sense. The efficiency analysis and visualization is intuitive.

Weaknesses

1. The paper mainly introduces two filters. The connection and difference between the two filters is not highlighted, which makes me curious about which one is better. 2. As far as I know, there exist many frequency filters in signal processing. The motivation for these two filters is not very clear. 3. In experiment part, the results of TexFilter on ETTm1, ETTm2, ETTh1 and Exchange are neither the best nor the second best. Does it show the filter cannot win in many cases of forecasting? Are there any analysis or explanation about it?

Questions

1. Can the authors elaborate on the connection and difference of the two filters? 2. Can the authors give more details about proposed filters? 3. Can the authors explain the lower performance of TexFilter in some cases?

Rating

7

Confidence

4

Soundness

4

Presentation

3

Contribution

4

Limitations

See weakness.

Reviewer UnVh7/10 · confidence 4/52024-07-08

Summary

This paper explores a novel direction that leverages frequency filters directly for time series forecasting, and proposes a simple yet effective framework, namely FilterNet, from a signal processing perspective. It designs two types of learnable frequency filters corresponding to the linear and attention mapping widely used in time series modeling, i.e., PaiFilter and TexFilter. Abundant analysis experiments verify the competitive modeling capability of frequency filters, and extensive experiments on eight real-world datasets demonstrate FilterNet achieves good performances in both accuracy and efficiency over state-of-the-art methods.

Strengths

1. This paper is well-written overall, and the contribution and novelty of the proposed work are clearly presented. 2. This paper introduces a novel and straightforward framework based on the frequency filters for time series forecasting. It offers a new direction by incorporating more signal processing theory into time series analysis. 3. The frequency filter has a simple architecture but demonstrates strong temporal modeling capabilities for time series. Results in Tables 1, 4, and 5 show that the two types of proposed frequency filters are suitable for different datasets and settings, which provides valuable insights for future model design. 4. This paper seems to have solid and extensive experiments. The experiments are conducted on eight real-world different datasets, and abundant evaluation analysis and visualization experiments are presented.

Weaknesses

1. The visualization setting for Figure 1 appears to be missing in Appendix C.4, and I recommend adding some references about the spectral bias of neural networks, such as [1]. 2. Some typo errors. In line 282, the complexity of FilterNet should be O(LogL), not O(L LogL).

Questions

1. In ICML 2024, some works also adopt simple architectures for time series forecasting, such as SparseTSF [2]. Could you further compare the performance of FilterNet with SparseTSF? 2. The recent literature has seen papers based on foundation models for time series forecasting. Conversely, some works are designed with simple architectures, yet achieve better performance than foundation models. Why can simple models (e.g., FilterNet) also achieve competitive performance? [1] On the Spectral Bias of Neural Networks, in ICML 2019 [2] SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameters, in ICML 2024

Rating

7

Confidence

4

Soundness

4

Presentation

4

Contribution

4

Limitations

N/A

Reviewer 2waa3/10 · confidence 4/52024-07-11

Summary

The paper proposes to use Fourier transforms with parametrized filters as layers for time series forecasting models. Filters can either be learnable parameters directly (called "plain shaping filter") or be a function in the inputs, gates (called "contextual shaping filter"). In experiments the authors show that their method outperforms many baselines from the literature.

Strengths

s1. two simple, but promising ideas: i) use Fourier transforms to represent time series and ii) represent the filters by gates. s2. well written paper s3. standard experiment with improvements over several baselines

Weaknesses

w1. Empirical comparison with results from a closely related baseline, FiLM are not shown, and FiLM reports better results for all but one dataset. - FiLM reports better results for ETTm2, Exchange, and Traffic. The proposed FilterNet wins for Weather. Why do you not report those results? If FiLM overall is the better performing model, why is FilterNet still interesting? How is it simpler than FiLM? Is it maybe faster to train? w2. Conceptual differences to existing layers using Fourier transforms are not discussed in detail. - So far, we just learn that other architectures also use "other network architectures, such as Transformers". But a more detailed delineation would be useful, esp. as FiLM is not using Transformers. minor comments: - reference [30], FiLM, is missing its venue, NeurIPS.

Questions

Q1. Why do you not report the better results of FiLM? Q2. If FiLM overall is the better performing model, why is FilterNet still interesting? How is it simpler than FiLM? Q3. Is it maybe faster to train?

Rating

3

Confidence

4

Soundness

2

Presentation

3

Contribution

2

Limitations

yes

Authorsrebuttal2024-08-12

Dear Reviewer 2waa, We appreciate your time and effort in reviewing our paper. As the discussion period is drawing to a close, we wanted to kindly inquire if there are any further concerns. Your feedbacks are really important to us, and we are also looking forward to further discussions with you before the discussion phase ends. We value your input and would appreciate the opportunity to respond. Thank you again for your valuable time and effort. Best, Authors

Authorsrebuttal2024-08-13

Dear Reviewer 2waa, With the discussion period nearing its end and less than 18 hours remaining, we kindly urge you to respond to our rebuttal.

Reviewer ufqs4/10 · confidence 4/52024-07-30

Summary

This paper investigates an interesting direction from a signal processing perspective, making a new attempt to apply frequency filters directly for time series forecasting. A simple yet effective architecture called FilterNet is proposed, utilizing two types of frequency filters to achieve forecasting. Comprehensive empirical experiments on eight benchmarks confirm the superiority of this paper's proposed method in terms of effectiveness and efficiency.

Strengths

1. The paper is in good writing shape and easy to follow. 2. The proposed method seems technically solid. 3. Extensive experiments on many datasets have been conducted to verify the effectiveness of the approach.

Weaknesses

1. The marginal improvement over existing approaches, as shown in Table 1, raises doubts about the compelling rationale for adopting the proposed method. 2. The discussion on related work appears lacking. It is common knowledge that frequency filters are widely used in traditional time series analysis methods and signal processing. Therefore, a detailed exploration of the connections and distinctions between them is warranted in the related work section. 3. The experiments have space to improve. Including more recent baselines for comparison, particularly deep learning methods that involve frequency processing, would add value. Additionally, a thorough investigation into the impact of hyperparameters is missing from the study.

Questions

Please address the questions in the weaknesses.

Rating

4

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

No

Authorsrebuttal2024-08-12

Dear Reviewer ufqs, Thank you very much again for your time and efforts in reviewing our paper. We kindly remind that our discussion period is closing soon. We just wonder whether there is any further concern and hope to have a chance to respond before the discussion phase ends. Many thanks, Authors

Reviewer ufqs2024-08-12

Reviewer Response

Thank you for addressing my questions. I still have the following concerns: 1. The authors have restated the technical contributions and efficiency achieved by the proposed method; however, the improvement in predictive performance is **neither significant nor consistent across all datasets**, particularly on the Traffic dataset. How many experimental runs were conducted for the reported results? From my understanding, the results of deep learning algorithms can vary across different runs. Can you guarantee that your improvements are statistically significant across all datasets? Additionally, what are the standard deviations across all runs? 2. The performance of TexFilter and PaiFilter varies significantly across different datasets. What is the underlying reason for this fluctuation? It’s difficult to determine when to use TexFilter versus PaiFilter. A more rigorous justification or empirical study is needed. Otherwise, I would prefer a model that demonstrates consistent performance across all scenarios, such as PatchTST and [1, 2]. 3. Although the authors provide several results comparing FITS and iTransformer, more recent and powerful baselines, such as TimeXer [1] and Timer [2], should also be considered. 4. Why didn't the paper compare the efficiency of frequency-based methods (e.g., FITS, FilM) with the proposed method? Compared to other types of models, it is more necessary to compare it with them. Moreover, it would be beneficial to include TimeXer and Timer in the efficiency comparison as well. Given the above concerns, I don't believe this paper is ready to publish on a top-tier venue like NeurIPS. Reference: [1] TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables. arXiv 2024. [2] Timer: Generative Pre-trained Transformers Are Large Time Series Models. ICML 2024.

Authorsrebuttal2024-08-12

Thanks for your feedback. We provide a point-by-point response to your new concerns below. **1. ....however, the improvement in predictive performance is neither significant nor consistent across all datasets, particularly on the Traffic dataset.** A1. We think your comment is not professional about improvements in the domain of time series forecasting. We don't understand what level of improvement is considered sufficient in your opinion. As you mentioned, TimeXer, what you refer to as a more recent and powerful method, its performance is worse than iTransformer on the ECL, Traffic, and Weather datasets, and worse than RLinear on ETTh2 dataset (please see its Table 8). Similarly, iTransformer shows weaker performance on the ETTh and ETTm datasets. Are you also questioning their performance improvements, too? It is **ridiculous** the failure to achieve optimal performance across all datasets could also be a reason of rejection. **2. How many experimental runs were conducted for the reported results? From my understanding, the results of deep learning algorithms can vary across different runs. Can you guarantee that your improvements are statistically significant across all datasets? Additionally, what are the standard deviations across all runs?** A2. Your question seems to be not related to our submission. We are wondering if you generate this review comments by Generative AI (e.g., ChatGPT). If you have run the state-of-the-art baselines in the domain of time series forecasting (e.g., iTransformer, PatchTST), you would know these forecasting methods are usually stable thus they did not report std. We conducted the experiments three times with three seeds, and reported the average performance with the very stable results and less std. Thus we followed previous settings to report the performance. The average improvement of FilterNet over all baseline models is statistically significant at the confidence of 95% (**please see in line 218 of our paper**). **3. The performance of TexFilter and PaiFilter varies significantly across different datasets. What is the underlying reason for this fluctuation? It’s difficult to determine when to use TexFilter versus PaiFilter. A more rigorous justification or empirical study is needed. Otherwise, I would prefer a model that demonstrates consistent performance across all scenarios, such as PatchTST and [1, 2].** A3. In the literature, compared with other model architectures, MLP-based and Transformer-based methods have achieved competitive performance. Inspired by the two paradigms, we design two corresponding types of filters. i.e., PaiFilter and TexFilter. PaiFilter offer stability and efficiency, making them suitable for tasks with static relationships. In contrast, TexFilter provides the flexibility to capture dynamic dependencies, excelling in tasks that require context-sensitive analysis. As shown in Table 1, PaiFilter performs well on small datasets (e.g., ETTh1), while TexFilter excels on large datasets (e.g., Traffic) due to the ability to model the more complex and contextual correlations present in larger datasets. This dual approach allows FilterNet to effectively handle a wide range of data types and forecasting scenarios, combining the best aspects of both paradigms to achieve superior performance. **We have analyzed the two filters in Appendix B and have also given explanations about the results of Table 1 in lines 219-224**. You again mentioned a **ridiculous** point. We don't know why the proposed method should be one model. Like DLinear, it also has studied three different variants, i.e., NLinear, Linear, and DLinear. Can you say you do not want to see the Linear family models in one paper? **4. ... comparing FITS and iTransformer, more recent and powerful baselines, such as TimeXer [1] and Timer [2], should also be considered.** A4. We have also compared FilterNet with SparseTSF (ICML, 2024) and Koopa (NeurIPS, 2023), and we are very confused that why we should compare with TimeXer which as only one preprint paper, and Timer as a generative pre-trained large model. **What is even more crazy** is that you recommend we include Timer in the efficiency comparison. Timer is a pre-trained model, which has a 28B pre-trained parameters (see its Table 9). It is brutal. **5. Why didn't the paper compare the efficiency of frequency-based methods (e.g., FITS, FilM) with the proposed method? Compared to other types of models, it is more necessary to compare it with them.Moreover, it would be beneficial to include TimeXer and Timer in the efficiency comparison as well.** A5. We have compared the efficiency with FiLM, please refer to A6 in Response to Reviewer 2waa. For FITS, it is a lightweight method and has 10k parameters. Obviously, FITS is more efficient. However, our model is not designed for a lightweight model, but a model that balances efficiency and effectiveness. As for the efficiency comparison with TimeXer and Timer, please refer A4.

Reviewer ufqs2024-08-13

Reviewer response

Thank you for the rebuttal. Here is my feedback: 1. Sure, the answer is not generated by GPT. The reviewer has worked in the field of time series analysis for many years since statistical methods. From a statistical standpoint, it is crucial or even compulsory to conduct rigorous statistical tests to evaluate improvements, especially when the improvements are insignificant. The reason you mentioned (other papers not reported) is not convincing and the evaluation goes against scientific rigor. 2. For instance, DLinear offers three variants, so why did the researchers only compare one? In future comparisons with your model, which variant should be selected? Which variant do you recommend? This is a very important problem in practice, especially for the industry. 3. The reason for not including TimeXer and Timer is reasonable. 4. My evaluation is centered on the technical novelty/contributions and practical results achieved by the proposed method. While the reviewer isn't particularly impressed by the technical novelty/contributions, the reviewer prioritizes methods that demonstrate strong performance in real-world applications. I have adjusted my ratings and would like to leave the discussion to ACs and other reviewers.

Authorsrebuttal2024-08-13

Thanks for your feedback. We provide a response to your main concerns below. 1. We will consider including the standard deviations in the final version, although none of our baselines—such as iTransformer, PatchTST, FEDformer, TimesNet, DLinear, RLinear, FITS, FreTS, MICN, Pyraformer, Autoformer, and Informer—report standard deviations in their papers. 2. In the literature, it is quite common to design different model variants to adapt to various datasets. For instance, FEDformer introduced two model variants, FEDformer-f and FEDformer-w; PatchTST designed two model variants, PatchTST/64 and PatchTST/42; DLinear included three model variants, DLinear, NLinear, and Linear. Switching between these variants typically involves adjusting a configuration parameter, such as the "version" parameter for FEDformer, "patch_len" for PatchTST, or "model" for DLinear. Typically, a model name is used generically to refer to all variants, with the specific variant being distinguished by its corresponding configuration parameters. For our proposed FilterNet, as mentioned in the previous rebuttal, TexFilter has demonstrated competitive performance on larger datasets (e.g., Traffic and Electricity) due to the need for more contextual structures, while PaiFilter performs well on smaller datasets (e.g., ETTh1). In practice, especially for the industry, the balance between efficiency and effectiveness is what matters most. And which model is more suitable requires actual comparison and testing. 3. As highlighted in our previous rebuttal, the primary contribution of this paper lies in offering a novel viewpoint from signal processing and devising a simple yet effective architecture centered around a Frequency Filter Block. FilterNet shows remarkable efficiency while simultaneously achieving state-of-the-art performance. Furthermore, we extensively investigate the efficacy and insights of frequency filters via introducing two distinct types of frequency filters, each exhibiting diversified performances across datasets of different scales. This exploration reveal the impact of frequency filters on time series forecasting and indicates that designing tailored architectures adaptive to data scale may enlighten a new path for future research endeavors.

Reviewer UnVh2024-08-13

Thank the authors for your rebuttal

Thank the authors for your rebuttal. I have reviewed all comments and the authors’ responses, which effectively addressed my concerns through clear explanations and additional experiments. I recommend incorporating these details into the manuscript or supplementary materials. I have also read the comments from other reviewers but I don't think their reviews can be too convincing to me to re-evaluate this paper. So I keep my original score and overall support the acceptance of this paper.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC