Summary
This paper proposes a star-shaped diffusion probabilistic model, which is non-Markovian, and more like an autoregressive way of predicting intermediate states xt. Specifically, the authors define a forward process q(xt|x0) that each intermediate state xt is directly related to the initial state x0. The reverse process is defined by p(xt|xt, xt+1, ..., xT). To efficiently solve p(xt|xt, xt+1, ..., xT), the authors propose a compressive representation Gt(xt, xt+1, ..., xT) to extract all the information of x0 contained in history states {xt+1, ..., xT}, so the modified distribution becomes p(xt|Gt). The training and inference go in a similar way as DDPM. Simple experiments show that the proposed method performs better than the baseline methods MTD and DDPM.
Strengths
1. This paper provides a novel view of the definition of diffusion models and is very inspirational. It is worth thinking about whether the Markov process is the best design choice considering the fact that the whole history {xt, xt+1, ..., xT} contains more information than a single state xt.
2. The author proposed a novel diffusion model with basic evaluations, which may inspire the community for further exploration.
Weaknesses
1. The presentation needs to be improved: (1) T, Ωt, and ht in Eq.13 lack interpretation; (2) It is not well proved that Gt covers all the information of x0 in {xt, xt+1, ..., xT}. Although the authors mention that there are explanations in the supplementary material, you need to ensure the reader has a basic understanding by just reading the paper.
2. The advantage and application value of the proposed method is not clear. For example, can you provide a practical case that DDPM can not address while SS-DDPM can?
3. The experiments are too simple and not persuasive. To prove the advantage of SS-DDPM, it is encouraged to compare with state-of-the-art methods on diverse benchmarks,
4. As the authors said in line 128, "In general case the dimensionality of Gt would grow with the size of the tail"
4. There is no evidence that SS-DDPM can apply acceleration methods like DDIM.
Questions
I doubt the reverse process of q(xt|x0) should be p(x0|xt), can you explain why the reverse process is p(xt|xt, xt+1, ..., xT)?
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Limitations
The scope of this paper is interesting, but it needs more time to polish the theory, presentation, and experiments. Considering its current form, I think this paper does not meet the acceptance criteria of NeurIPS. However, if there is enough evidence to show the value of this paper, I may also change my score.