Transparent Networks for Multivariate Time Series

Summary

GATSM effectively captures temporal patterns and handles dynamic-length time series while preserving transparency, outperforming existing GAMs and matching the performance of black-box models like RNNs and Transformers.

Strengths

* This paper is easy to understand. * GATSM can be understood as a linear representation with good transparency. * Surprisingly, this method improves performance while providing better interpretability compared with black-box.

Weaknesses

* It is not clear how multi-head attention works in Definition 3.1 to learn temporal patterns. My understanding is that the global feature interacts with the current feature in Eq.3, so why is it that the input to the attention is not $x$ but the transformed $\tilde_x$. And then, are temporal patterns captured from attention? * The addition of attention to NBM is under-motivated, so why not just replace $w^{nbm}$ to attention weight, so that you can learn one less set of parameters $w^{nbm}$. * The experiment lacks DLinear [1], a strong baseline and providing interpretability. I think it's highly relevant. 1. Are Transformers Effective for Time Series Forecasting? AAAI 2023. * Given the emphasis on interpretation or white-box modeling, qualitative experiments of the contributions/explanations need to be compared rather than visualization. If there is no ground-truth of the contribution, occlusion experiments in post-hoc methods [2, 3] can also be designed to explore the trade-off between performance and additive features. 2. Encoding time-series explanations through self-supervised model behavior consistency, NeurIPS 2023. 3. TimeX++: learning time-series explanations with information bottleneck, ICML 2024. * In Table 7, why is the throughput of GATSM so much lower than NBM? Does NBM add up as features at the time level without sharing?

Questions

See the weaknesses.

Rating

5

Confidence

4

Soundness

2

Presentation

3

Contribution

2

Limitations

It's hard to deal with the higher-order interactions, e.g. GA^2M/GA^NM in time series, since time series often have long time points and the high complexity of gam-based techniques. In addition, the temporal-level causal interrelationships can be further explored.

Summary

This paper introduces GATSM(Generalized Additive Time Series Model), designed for handling multivariate time series data with a focus on transparency and interpretability. Using independent networks to learn feature representations and transparent temporal modules to learn cross-time step dynamics, GATSM effectively learns temporal patterns and maintains interpretability. It achieves comparable results achieves comparable performance to black-box time series models on various datasets, and proves the transparent predictions with cases.

Strengths

1. A key strength of GATSM is its focus on transparency, providing clear insights into the decision-making process, which is crucial for applications in high-stakes domains like healthcare. 2. The paper presents a thorough evaluation across multiple datasets, including Energy, Rainfall, AirQuality, and several healthcare datasets, showcasing the model's robustness and generalization capabilities. 3. The authors provide extensive details about the experimental setup, including data splits, hyperparameters, and computational resources, facilitating reproducibility.

Weaknesses

1. The need for a large number of feature functions can limit scalability (even if it is reduced from TxM to B), particularly with high-dimensional data. 2. The dataset tasks encompass both 1-step forecasting and classification, each requiring distinct evaluation metrics. Presenting all results in a single table without proper clarification leads to confusion. 3. The selected baseline black-box models are relatively simple. Consider including one or two state-of-the-art methods for a more comprehensive comparison.

Questions

Can this model be applied to a multi-step prediction explanation since one-step forecasting is simple to real-world applications.

Rating

5

Confidence

4

Soundness

2

Presentation

2

Contribution

3

Limitations

Yes

Summary

The paper introduces a Generalized Additive Model for time series, combining feature embedding and attention layer. The proposed solution is evaluated on forecasting, binary and multiclass classification, over 8 datasets, against black box and transparent models. Global, local, time-focused and feature focused interpretability methods are provided.

Strengths

Experimental evaluation is convincing: the presented model has state of the art performance. On the transparency side, the comparison between models allows the author to postulate on the presence of absence of interactions of covariates, and time-specific patterns. Evaluations of weights at the final linear layer give varied visualizations. The reasoning behind the model construction and component is clear, and ablation experiment for each component were provided. Paper is clear with no major problem in writing.

Weaknesses

I am not sure if the work is original. The idea of using DNN first on the time axis without covariate interaction is not new, but wether there is a model similar to the proposed solution, I do not know. The review of previous works focuses on Generalized Additive Models, but a similar neural structure may have been presented without being positioned as a GAM. One improvement to be done would be to add more clarity on figure captions. In its present form, it requires back and forth to the text to understand both what is plotted and what conclusions to draw from it.

Questions

In forecasting tasks, there are often long horizons to be considered. Is it possible to build a multioutput model for forecasting using GAM formalism? In which case, would the interpretation be able to relate future timestamps together, as series are often autocorrelated? Can correlation among dimensions be detected with GAM? This would be especially relevant in high dimensional MTS.

Rating

6

Confidence

1

Soundness

3

Presentation

3

Contribution

2

Limitations

Several limitations are identified: possible overfitting of the model due to overparameterization in NBM part, slow attention mechanisms that do not benefit from state of the art methods, and the fact that it was not evaluated for long sequences (and might not be suited for them).

Summary

This work aims to build transparent models for the time series domain for better interpretability. Specifically, they proposed a work called Generalized Additive Time Series Model (GATSM) that consists of independent feature networks as well as a temporal attention module to learn temporal patterns. The corresponding model can be written into a scalar form to ensure interpretability/transparency. The authors applied their model to several datasets, and showed that GATSM can outperform existing generalized additive models. The model can also be used to interpret the features in original dataset.

Strengths

- Important motivation. Building transparent models for time series is a critical task. - The scalar representation of features seems clean (eq 11)

Weaknesses

While this work is not of my direct expertise, I think the following contents have room for improvement: 1. Experimental results are weak. - Black-box Time Series Models seem out-of-dated. The authors should consider better models such as TimesNet, PatchTST, FreqTransformer, or Informer for commonly used black-box models. - Forecasting tasks use R2 score for evaluation, but an R2 score of 0.07 (or in general, below 0.5) seems very low. The authors should show some visualization examples to ensure the model is functioning. - Figure 4/5 are not self-explainable, the authors should try to explain what is happening in those figures, and how the interpretability is quantified. - The work could benefit from synthetic dataset, where casual relationships are manually crafted and thus can be evaluated.

Questions

- Line 123: How one should understand the proposed basis functions, are they similar to quantized vectors in works like VQVAE? - Eq 5: How would positional embeddings affect the interpretability of the model? - How does the interpretability results differ in datasets with very long length (e.g. Mortality) v.s. that of shorter length?

Rating

4

Confidence

2

Soundness

2

Presentation

3

Contribution

2

Limitations

Yes