Summary
The authors address the issue that most real-world time-series are non-stationary. To tackle this problem they introduce Koopa, a globally learned (time-invariant) and localized (time-variant) linear Koopman operators, to exploit respective dynamics underlying different components. In practice, Koopa is composed of stackable Koopa Blocks, where each block learns operators hierarchically by taking the residual of previous block fitted dynamics as input. The authors evaluate the proposed model on variety of time-series data sets and compare their method with transformer and convolution based deep architectures. Koopa outperforms or performs on par with the existing methods, yet has a significantly lower computational costs. Moreover, the proposed method can be use in settings where the forecast horizon must be increased beyond the training set-up.
Strengths
Originality:
The related work section in well written. It covers both relevant model types in the field, for (a) time series forecasting, TCN, RNN and transformers are mentioned, as well as the authors position the current work in its context; for (b) Koopman operators, the authors provide an overview of the related work and how their work differs.
The work proposes a novel method leveraging Fourier analysis to disentangle time-series components, eDMD for estimating time-variant Koopman Predictor, and time-invariant Koopman Prediction as a learnable parameter. It differentiates itself from KAEs due to its block like structure, where the residual of the previous block is fed as input into the next block.
Quality:
The submission is techinncally sound. The authors build upon Koopman operator theory and Wold's Theorem. In addition, experimentally they validate whether their proposed sub-components of Koopa, Fourier Filter, truly disentangles time-variant from time-invariant series. As well as provide an ablation on Koopa structure.
Clarity:
The work is well written providing sufficient relevant background knowledge on Koopman operator theory also for reader who is unfamiliar with this theory.
Significance:
The obtained results are significant. The proposed method outperforms or performs on par with existing time-series forecasting methods. More importantly, the method is very efficient with respect to training time and memory costs compared to transformer or convolution based methods. Lastly, the authors showcase that their method is applicable for scaling up forecast horizon, where standard deep learning architectures fail. In all, based on the submission the author's suggest a very promising modelling technique, however, I am not an expert of Koopman operator theory (nor the works in this sub-domain).
Weaknesses
Clarity:
When first mentioned, I would recommend using the complete name of the model type rather than only the acronym, for example, temporal convolutional network rather than just TCN, it makes it easier for the reader to read.
Questions
For time-variant KP how do you choose S, is the Time-variant KP sensitive to S?
Rating
8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
Confidence
2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Limitations
The authors have not addressed limitation of their work. I would suggest the authors to include such section in the final version of their work.