Summary
This paper proposes GLAFF which encodes the time stamps of time series and performs self attention across the encodings of the time dimensions, combining it with the output of a global time series forecaster via a learned weighting scheme. As GLAFF is a set of feature constructions, it’s generally additive in performance against any backbone architecture. The main requirement is that GLAFF seems to require knowledge of when the prediction should be produced for.
More specifically, the time stamps themselves are used to generate de-medianed and de-quantized predictions. This is extra side information, and so it should generally tend to be helpful – some of the tested datasets have clear time information. For example, traffic tends to spike in the morning and afternoon hours, while electricity also has daily and monthly peaks and valleys. This is a natural encoding of seasonality and other similar regular events.
Strengths
Originality:
- Most other papers in this literature tend to focus primarily on architecture, and the ones that do take into account some sort of feature information seem to not incorporate it that well.
- Overall, I like the idea of separating the time encoding into what’s essentially its own network.
Quality:
- I appreciate the knockout studies of the varying parts, which is good experimental design.
- I also appreciate the knockout study of iTransformer and would like to see more of these types of results.
Clarity:
- Overall, the paper is pretty well written. It’s pretty much clear what’s going on.
Significance:
- Improving time series forecasting is obviously a highly important problem, and improving the base model is a general purpose technique that should generally be quite useful.
Weaknesses
Quality:
- My biggest concern in terms of usefulness (and unfortunately, this is somewhat a critique of the entire vein of literature here) is that the time stamps of the prediction window are trained on. This can be, in some sense, a pretty strong lookahead bias. In most deployment settings, we cannot pre-train on the time stamps that we’re going to use to generate the sequence because we do not know the absolute value for the time stamp during train time.
o Thus, the method is limited to explicitly semi-regular time series where the forecast clock is known ahead of time. By non-regular forecasting, we can think of wanting to predict the inventory levels of a product after the next sixty sales, or to predict the price of a foreign currency after some amount of trading volume in it. Instead, in regular forecasting, at the start of the day, we may wish to predict the next day’s electricity demands (one of the benchmark datasets).
Clarity:
- I personally find $\tilde X$, $\hat X$, $\hat Y$ to be quite confusing in terms of notation. Perhaps $\hat T$, $\tilde S$, $\hat S$ could be used instead throughout to denote the fact that these values come from the encoded time stamps.
o In line 197, I would encourage against setting the final prediction as $Y$, but rather as $\hat Y$.
- I also find the methods section somewhat unclear, as it’s a bit tricky to parse out that the median / quantile stats are dependent on the input time series, X. Perhaps it would be better to not separate X and Y entirely, but rather point out that the time series is really cat(X, Y).
- From the intro and contributions, it could probably be made clear that other papers _do_ consider time stamp information; however, it doesn’t seem to be helpful in knockout studies. This is more of a writing note than something major.
Questions
- In Eq 3, where do the median and quantiles of Y come from? Do they come from the global set of sequences? This is I think more for clarity at this point but want to confirm.
- It seems that most of the improvement in exchange and weather, which are not periodic, is driven by improvement in the worst model (informer). In general, should we expect that there is less improvement on aperiodic data because the time embeddings are less helpful?
- Often in time series prediction tasks, there is other side information, is this naturally best situated to being encoded solely inside the backbone architecture or should it be modeled alongside the time stamps?
- Tables 1 and 2: I’d suggest using bar charts to make the presentation more engaging, and moving the tables to the appendix.
- I find it fairly surprising that informer, timesNet, and iTransformer tend to have minimal dropoff when removing the timestamp pieces. Is this lack of dropoff consistent across datasets?
- Could the authors include an average improvement across methods for a fixed horizon? It would be interesting to see if DLinear improves the most as a result of adding in the time step information.