Predict, Refine, Synthesize: Self-Guiding Diffusion Models for Probabilistic Time Series Forecasting

Summary

The paper focuses on generative diffusion modeling when some of the samples are known, which is advocated as the right setup for regression. Indeed, in a time-series context, those known samples may lie in the past (prediction), or provide boundary conditions (interpolation). In this context, the classical diffusion equations can be worked out to yield a generation that is compatible with the known samples to some controllable extent. This is the key contribution for this paper, and I must say that traming regression as a constrained diffusion is an interesting idea. From this starting point, the authors try to derive two variants: one where the model estimate is used as a prior, and one where some arbitrary other estimate is used. A set of forecasting experiments conclude the paper

Strengths

- the paper is very readable and provides some background and references. It is quite clear - Framing regression as a constrained diffusion is definitely interesting and has some merits. - I would also say that I'm sure that the proposed method yields interesting results, compared to some other methods

Weaknesses

Notwithstanding its qualities, I have several concerns with the paper that prevent me from giving it a high score. - I would say that the attempt at formulating the method in a Bayesian way is actually a bit awkward in my opinion, because it does not bring much more food for thought than just plainly presenting the guidance as a regularization over the (enegry-based) generating process. Indeed, the prior distribution that is picked in equation (5) just reads like picking some L2 regularization to me, and the quantile regularization that is advocated as working very well in practice just doesn't lend itself very well to a convenient Bayesian treatment. Actually, section 3.2 precisely drops this Bayesian vision to adopt an optimization-based point of view, which seems more natural to me. - I feel that there is actually a lot of redundancy between 3.1 and 3.2. As I see it, the two "variants" are actually the same, with the only difference that for self-guidance, the estimate is derived from the model while it is assumed arbitrary in 3.2.

Questions

- L85: consists of decomposing - L119: "see Eq 1": It is not clear to me why you provide this reference to eq (1) - eq (5), again, the choice of a unit variance for the likelihood is weird to me. I would say this is only picked this way to motivate some L2 regularization term in a "reverse mathemating" sense - I don't understand how you get \epsilon in equation (10) and further during inference time. - L249-250: when you write that the model does not require task-specific training, could you actually explicitly mention *what* is the model trained on ? I thought that there would be a separate training for each task ?

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

- will you provide some implementation ?

Summary

This paper proposes TSDiff, which is an unconditional diffusion model for time series generation. Besides, the authors propose self-guidance and prediction refinement. The empirical results showcase the superiority of TSDiff over existing baselines on forecasting, refinement, and generating synthetic samples.

Strengths

1. The paper is joyful to read and the methodology is clearly described. 2. The empirical results are more than enough. The authors conduct extensive experiments on various of tasks and datasets, making the proposed methodology very convincing. 3. It is good to see that the authors proposed a new metric to make more reasonable comparison with other baselines in Section 5.3.

Weaknesses

1. If y_obs is a subset of timesteps representing the observed timesteps, then is the conditional generation similar to an inpainting problem in 2D images? Then important baselines in solving inverse problems with pre-trained diffusion models, e.g., DDRM [1], should be included, or at least be discussed thoroughly. 2. There are several equations that are not labeled with numbers. It would be the best if the authors can label them. I have several questions regarding the equation after Eq. (10): [1] Is tau hard to solve? It looks like solving tau is computationally expensive. It would help if the authors can provide some empirical results. Actually I did not quite understand "tau can be computed efficiently by tracking the running mean for the losses at all diffusion steps during training". [2] How does the optimum, tau, vary with different data point? In this equation, the expectation does not take x_t (or y) into account, does this mean we have to solve for tau for every y? Or do we jointly solve a tau using all the data points in the training set?

Questions

See Weaknesses.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The limitations are well discussed in the paper.

Summary

This paper describes a diffusion model for time series problems. Contrary to the popular approach of using a conditional diffusion model, the authors proposed to use the unconditional diffusion model, which is supplemented by a self-guidance mechanism. The authors also proposed a prediction refinement algorithm to improve the prediction of any time series model by using the trained diffusion model.

Strengths

Training a conditional diffusion model is challenging due to the high dimensionality of the conditional problem. The authors proposed an interesting method of bypassing training a conditional diffusion model. The proposed methodology is simple and sound.

Weaknesses

The paper does not elaborate the details of their model, which makes it difficult to fully understand the methodology. For example, it does not clearly show the dimensions of input and output variables, how the missing inputs are treated, and so on. There are also a few comments in Questions sections.

Questions

1. In equation (6), as $t$ gets larger, the denominator $\sqrt{\overline{\alpha}_t} \rightarrow 0$, which is a natural consequence of the diffusion process. How do you handle this singularity issue? In practice, it will not be singular, but potentially cause a numerical instability. 2. The target distribution, $p(y_{ta}|y_{obs})$, but it actually should be $p(y_{ta}|y_{obs}) = p(y_{unobs}|y_{obs})p(y_{obs}|y_{obs})$ and $p(y_{obs}|y_{obs}) = \delta(y-y_{obs})$. So, upon the guidance denoising, we expect the observed $y$ should be converged to $y_{obs}$. I wonder if the model guarantees this. 3. Provide the proof that equation (9) converges to the maximum likelihood solution when $\gamma = 0$. Also, please provide a proof that (9) converges to $p(y)$. 4. Carefully reading it, I can understand how the guided diffusion, eqn (5), is computed for missing data and prediction. But, it will be useful if the authors can elaborate it to improve the readability. 5. It is claimed that the model can be used for imputation. Is the diffusion model trained with a missing data? Or, does it require a full data? *I am willing to upgrade my score once these questions are addressed.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

One of the questions of time series diffusion model is whether it memories the pattern or learns the dynamics. Based on the learning objectives, it is more likely to learn the patterns, which may limit the capability of diffusion-based models for the time series problems of chaotic dynamics and a stochastically forced system. Also, as the authors suggested, the computational cost is certainly of concern for a scalability. But these are beyond the scope of the study.

Summary

In this paper, the authors proposed an unconditional diffusion model and a self-guidance mechanism for time series data that can be used for conditioning diffusion model for downstream tasks, e.g. time series forecasting and imputation. The effectiveness of proposed model is evaluated from three aspects: prediction, refinement of prediction from other base forecaster, and time series generation. The results show that unconditional diffusion models can achieve comparable forecasting accuracy to conditional ones. The proposed model can be potential useful for other downstream time series analysis tasks other than studied ones.

Strengths

1. The author proposed a new self-guidance mechanism to approximate conditional probability p(y_obs|x^t) which enables time series forecasting with unconditional diffusion model 2. Extensive experiments on multiple benchmarks demonstrate the effectiveness of the proposed unconditional diffusion model comparing to conditional ones. 3. The paper is well written and easy to follow.

Weaknesses

1. Refining the prediction of a weak forecaster is a good way to evaluate the model, but it has no real application scenario.

Questions

1. The authors described the details of how to do self-guidance without explanations of why this approximation can work in practice. It would be helpful to explain the high level intuition behind it. 2. The idea of self-guidance seems to be general, can it be used in other domain, e.g. image? 3. It is a good way to evaluate the model with refining the prediction. But this is not a practical senario. If one has the diffusion model, why not just use it for prediction? There are a lot of spaces used for this in section 4. It would be better for me to discuss other more interesting things. 4. The authors mentioned that the computational cost of the proposed model is high. It would be better to have such experiments comparing the forecasting speed with conditional diffusion models. 5. The authors evaluate the time series synthesis using train on synthetic-test on real setup. It would also be interesting to see train on synthetic and real - test on real setup. Can synthetic time series helpful for forecasting? 6. The experiments are conducted on univariate case, can the method be applied for multi-variate time series? 7. Can other time series analysis task get benefit from proposed models other than forecasting and imputation? For instance, time series classification, clustering, similarity search.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Yes, the author claimed the limitation of the paper, which is the computational cost.

Thank you

Thank you for the additional results and clarifications. I will keep my rating and I tend to accept this paper.