Weaknesses
1. Although the proposed method can model/forecast periodic signals, I am not convinced it outperforms state-of-the-art RNNs [2] or other successful methods for analyzing and modeling time series data, such as Koopman-based methods [1]. These methods can model and forecast not only periodic time series but also more challenging time series data related to chaotic systems, see for instance [1,2,3,4,5].
- Could the authors compare more aspects, for example, **training time** and the **hyperparameters** (the number of hyperparameters and computationally consuming hyperparameter search) for their method and the state-of-the-art methods (e.g. shPLRNN in [2]) for simple benchmark systems like the van der Pol oscillator?
Also, please note that while MSE seems an appropriate evaluation measure for periodic time series, for non-periodic time series data, one should consider other measures like Geometrical Measure (D_stsp) and/or Temporal Measure (D_H) as mentioned in [2].
2. While periodic signals are indeed significant in various applications, real-world datasets often exhibit a wide range of behaviors, including non-periodic, chaotic, and irregular patterns. Therefore, designing a method that exclusively targets periodic signals may limit its applicability in practical scenarios where the underlying dynamics of the time series are **unknown** or **complex**.
- Could the authors test NeRT on non-periodic time series data, for example, on chaotic systems like the Lorenz or Rössler systems, to demonstrate the broader applicability of their method?
3. The paper mentions comparisons with RNNs, but it does not specify which types of RNNs were used in the experiments (in addition to LSTM). In recent years, various RNNs with specialized training algorithms have been developed to capture, model, and forecast time series data. Thus, to ensure a fair and comprehensive evaluation of NeRT's performance, the authors should include comparisons with these advanced RNN architectures, which are designed to capture **different types of temporal dependencies** in time series data. By benchmarking against **state-of-the-art RNNs** (e.g. [1,2,5]), the authors could provide clearer context for the advantages of their approach, thereby strengthening the validity of their claims regarding NeRT's superiority in handling periodic signals.
- Could the authors compare NeRT with some state-of-the-art RNNs, e.g., shPLRNN in [2], AL-RNNs in [5] or/and Sampled RNN in [1]?
---------------------------------------------------------------------
**References**
[1] Bolager, Erik Lien, Ana Cukarska, Iryna Burak, Zahra Monfared, and Felix Dietrich, "Gradient-free training of recurrent neural networks", arXiv preprint arXiv:2410.23467 (2024).
[2] Florian Hess, Zahra Monfared, Manuel Brenner, and Daniel Durstewitz, Generalized Teacher Forcing for Learning Chaotic Dynamics, In Proceedings of the 40th International Conference on Machine Learning. PMLR, (2023).
[3] Georgia Koppe, Hazem Toutounji, Peter Kirsch, Stefanie Lis, and Daniel Durstewitz, Identifying nonlinear dynamical systems via generative recurrent neural networks with applications to fmri, PLoS computational biology, 15(8):e1007263, (2019).
[4] Jonas Mikhaeil, Zahra Monfared, Daniel Durstewitz, On the difficulty of learning chaotic dynamics with RNNs, Advances in Neural Information Processing Systems, 35:11297-11312, (2022).
[5] Brenner, Manuel, Christoph Jürgen Hemmer, Zahra Monfared, Daniel Durstewitz, Almost-Linear RNNs Yield Highly Interpretable Symbolic Codes in Dynamical Systems Reconstruction, NeurIPS 2024, arXiv preprint arXiv:2410.14240 (2024).