Piecewise deterministic generative models

Summary

This paper proposes a new generative model on continuous random variables similar to diffusion-based generative models, but this method uses a different family of perturbation schemes. In particular, for the forward processes, which transform the clean data to a stationary distribution, the authors propose using piecewise deterministic Markov processes (PDMPs), which consist of a combination of discrete diffusions and ODEs. Specifically, the initial value, i.e., data, is transformed via simple ODEs, whose velocity term is often chosen to be a constant. However, the velocity term (also called vector fields) is updated at any random time with a constant rate $\lambda$. The occurrence of a new event is independent, and the time interval of the new event since the last one is exponentially distributed. Moreover, the new velocity will be assigned by user-chosen schemes; for example, one can flip the sign of the velocity, or another one can sample from the standard normal distribution. Thus, while the forward process transforms the data with a simple ODE, it randomly changes the ODEs, hence, the title of the paper. In addition, the paper shows that under some mild assumptions, the time-reversed processes of the PDMPs are also PDMPs, and their transition probability distribution and update rate can be written in terms of the forward processes. Therefore, we can consider modeling such terms with parametric models similar to popular diffusion-based generative models. The paper proposes the explicit ratio matching objective function, as in Equation 6, to train such terms. Moreover, the authors suggest using three popular PDMPs for the forward process: Zig-Zag process (ZZP), bouncy particle sampler (BPS), and randomized Hamiltonian Monte Carlo (RHMC). Consequently, the paper shows the time-reversed PDMPs and potential parameterizations that fit each case. Finally, the paper demonstrates the efficacy of the proposed method through a few toy experiments. ---------------------------------- Update the rating from 5 to 7 after the authors' rebuttal.

Strengths

One of the paper's key contributions is the introduction of a novel generative model that uses PDMPs as the forward processes. Moreover, the authors propose a training method for the time-reversed PDMPs.

Weaknesses

The proposed method is novel and interesting. However, the paper didn't characterize some potential drawbacks well. For example, due to the nature of PDMPs, the training of the proposed method inherits the problem of discrete diffusion models. In particular, the ratio matching should be done for each dimension independently, and this costs a lot for high-dimensional data compared to the denoising score matching and others for previous methods. This problem could be the reason that the current submission didn't include the experiments on popular high-dimensional datasets. In this regard, the paper suggests that Monte Carlo estimates can reduce the computational cost, but it trades off the variance of the learning signals, which would be critical for large-scale experiments. I consider that a new method doesn't always need to be better than previous methods. Nevertheless, proper information about the proposed method in practice would be important for potential readers to evaluate the significance of the paper. Thus, some additional discussions related to this problem need to be added. Moreover, given that many alternative choices exist other than the proposed method, in-depth discussions on the proposed method's merits would be appreciated for evaluating its importance. In addition, I find that the presentation should be improved. For example, many statements contain multiple prepositional phrases, which are difficult to parse and understand.

Questions

N/A

Rating

7

Confidence

4

Soundness

3

Presentation

2

Contribution

3

Limitations

N/A

Summary

The paper explores generative models utilizing PDMPs, a type of stochastic process characterized by deterministic motion interspersed with random jumps at random times. These models offer an alternative approach to diffusion-based generative models, which have become very popular in the AI community in recent years. The authors focus on three specific PDMPs: the Zig-Zag process (ZZP), the Bouncy Particle Sampler (BPS), and Randomised Hamiltonian Monte Carlo (RHMC). The authors leverage the existing literature on the time reversals of Markov jump processes, characterizing the time reversal of any PDMP under appropriate conditions. They show that the time reversal of a PDMP is itself a PDMP with characteristics related to the original process. The authors also specifically outline the characteristics of the time-reversal processes for ZZP, BPS, and RHMC. The jump rates and kernels of these time reversals admit explicit expressions based on the conditional densities of the PDMPs before and after jumps. The authors provide bounds on the total variation distance between the data distribution and the distribution of their generative models, considering errors from the approximation of the backward PDMP’s characteristics and its initialization from the forward process’s limiting distribution. Some initial but promising numerical simulations on simple toy distributions are presented, showcasing the potential of PDMP-based generative models. The results support further investigation into this class of models on more challenging data structures.

Strengths

Originality: The paper introduces a class of generative models based on piecewise deterministic Markov processes (PDMPs). This is a very novel idea which departs from the widely used diffusion-based models. In my opinion, this is a fresh perspective on generative modelling and opens up new avenues for exploration and potential improvements in various application areas. The key idea behind diffusion-based models is the derivation of the reverse time and process, and following a similiar line of thinking, the authors of this paper characterize the time reversal of PDMPs, which is particularly original. While time reversal in diffusion processes is well-studied, applying these concepts to PDMPs and providing explicit expressions for their jump rates and kernels is a significant contribution. Quality: The paper presents a comprehensive theoretical framework for PDMP-based generative models. The mathematical rigour in characterizing time reversals, deriving error bounds, and proposing training procedures leads to a complete and high-quality piece of work. The thorough analysis of the three specific PDMPs (ZZP, BPS, RHMC) and the detailed exploration of their time-reversal characteristics demonstrate a deep understanding and careful consideration of the underlying processes. The numerical simulations provide supporting empirical validation of the proposed models, adding credibility to the theoretical claims. The use of multiple toy distributions (extras are in the appendix) to test the models in the paper strengthens the evidence for their potential efficacy in practice. Clarity: The paper is very well-structured, with a logical flow from the introduction of PDMPs to the detailed theoretical contributions and empirical results. Each section builds on the previous one, making it easier to follow the progression of ideas. The explanations of the PDMPs, their time reversals, and the derivation of jump rates and kernels are clear and very detailed. The inclusion of propositions and their proofs in the appendices provides a robust foundation for the proposed algorithms and establishes their convergence properties. The use of toy distributions to demonstrate the numerical simulations helps illustrate the practical application of the models, aiding in the reader’s understanding and also provides some nice opportunities for the authors to clearly articulate the advantages of their approach over the diffusion-based alternative approach. Significance: The proposed PDMP-based generative models have the potential to be applied in a wide range of fields, from machine learning and statistics to physics and biology. This broad applicability enhances the significance of the work. By offering an alternative to diffusion-based models, this paper advances the field of generative modelling. The potential advantages of PDMPs, such as better scalability and reduced computational complexity, could lead to significant improvements in high-dimensional data generation. The theoretical insights and empirical results lay a strong foundation for future research in this area.

Weaknesses

Originality: While the introduction of PDMP-based generative models is novel, the paper primarily focuses on theoretical aspects and toy datasets. There is a limited exploration of how these models can be applied to more complex, real-world problems, which could showcase their true originality and practical utility. Although PDMPs offer a new approach, the paper does not extensively compare these models with a wide variety of existing generative models, even on the simple toy examples that were considered here. This makes it difficult to fully appreciate the originality and benefits of PDMPs over other state-of-the-art methods. Quality: The paper relies on several technical assumptions and conditions (e.g., H3, H4). While these are necessary for the theoretical results, their practical applicability might be limited and the authors do not discuss whether or not these assumptions hold for the toy examples which they consider. The paper could be strengthened by discussing the feasibility of these assumptions in real-world scenarios. The empirical validation is primarily limited to simple toy distributions. While these are useful for initial validation, the lack of experiments on more complex datasets (e.g., image or text data) reduces the overall impact and persuasiveness of the empirical results. It would be beneficial to include comparisons on more challenging benchmarks. Clarity: The paper uses dense mathematical notation and detailed proofs, which may be challenging for readers who are not specialists in stochastic processes or PDMPs. This is going to be challenging for the authors as the high-level of technical detail provided in the paper does lead to a very robust paper. However, perhaps more intuitive explanations or visual aids could help make the content more accessible, if not in the main paper then in the appendix. The practical implementation details, particularly regarding the training procedures and simulation methods, are somewhat sparse (even though there are more details in the appendix). Providing a step-by-step guide or pseudocode could help practitioners better understand how to apply the proposed methods. Given the space constraints, this would have to be added to the appendix. This is covered in the case of splitting schemes, but could perhaps be modified to be more user-friendly to people new to this area of research. Significance: The paper’s significance is somewhat limited by the focus on theoretical and synthetic examples. Without demonstrating the effectiveness of PDMP-based models on real-world data, it is challenging to gauge their practical significance and potential impact in applied settings. While the paper claims that PDMPs offer better scalability and reduced computational complexity, there is limited empirical evidence to support these claims. Benchmarking the computational performance against existing generative models would provide a clearer picture of the advantages and limitations in terms of scalability and efficiency.

Questions

- I've outlined quite a few points in the weaknesses section that the authors may wish to address as questions. - Could the authors include computational time with their simulations? - Under H2, the authors state that the condition is satisfied when "the tails of $\mu_*$ are sufficiently light." Could this be clarified as to what "sufficiently light" means? I'm interpreting this as lighter than Gaussian. - A small query, in Proposition 2, on Line 166, it states that $\mu_0^X \otimes \mu_0^V$ on $\mathbb{R}^2d$. But how does this work for ZZP when $V=[-1,+ 1]^d$ (sorry - I can't get the curly brackets to work)?

Rating

7

Confidence

5

Soundness

4

Presentation

4

Contribution

4

Limitations

NA

Summary

This interesting paper on the popular topic of generative models introduce a new family of generative models which builds on the so-called piecewise deterministic Markov process (Zig-Zag process, Bouncy Particle Sampler, Randomised Hamiltonian Monte Carlo). In contrast to many of the existing models this family is not based on diffusion models. The paper includes a through analysis of the construction and it propose training procedures and methods for approximate simulation of the reverse process.

Strengths

* A new family of generative models is proposed. * Thorough analysis of the properties of the proposed construction is provided. * Simple examples provided.

Weaknesses

* Missing real-world examples * The is a big jump in the style of writing between Section 1 and 2. Do not get me wrong here, the technical developments are most interesting, but many readers would be helped by a more gentle transition between these sections. Space for this can be created by moving more of the technical details into the supplemental material.

Questions

* You provide convincing examples on simple synthetic datasets. Is you family of methods able to generate data also in more challenging real-world examples? * When you compare the results of the various methods you show results for "steps". Is the a fair way of comparing the methods? Would it not be better to have some kind of computational cost on the x-axis? My reasoning here is that a "step" is not a well defined concept and it could mean very different things to different methods. In any case it would be really interesting to see the performance with the computational cost on the x-axis.

Rating

6

Confidence

4

Soundness

3

Presentation

2

Contribution

4

Limitations

Real-world examples missing.

Summary

This paper considers the development of generative models based on piecewise deterministic Markov processes. The key idea proposed in the paper is to use piecewise deterministic Markov processes instead of diffusions as the "noising process" of the generative model. This relies on the fact that time reversals of PDMPs are themselves PDMPs. Three specific instances of PDMPs are considered. The authors also derive a bound (in total variation distance) between the data distribution and the distribution of the generative model. The methodology is illustrated with some simple experiments.

Strengths

I enjoyed reading the paper and I like the idea of considering alternative noising processes in the context of generative models. The paper covers both theory and provides an example showing the viability of these methods. The examples are sufficient and certainly the area seems worthy of further investigation.

Weaknesses

To me, the descriptions of approximating the process characteristics with normalizing flows are unclear. This part should be written more with more details, perhaps in the supplement, as this is core to being able to reproduce the results. I do appreciate that the authors provided a description of the experiment in E.1, but it is not enough to put things together. I would be interested in replicating at least the simple experiment, but I don't think I can do it as the paper stands.

Questions

Are there particular data characteristics that these processes would be most suitable for? It seems that images with sharp boundary transitions are better modeled by the authors proposed approach. Would you say that is the case? Do these methods have issues like being only able to generate from one mode if the data distribution is multimodal? Why does RHMC do so well for such a small number of steps?

Rating

7

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

Yes.

Summary

The paper proposes using Piecewise Deterministic Markov Processes (PDMPs) for generative modelling applications, by using the property that PDMPs also admit time reversals that themselves are PDMPs. There are three major contributions in my understanding - By characterizing certain families of PDMPs, i.e. Zig-Zag processes (ZZP, the Bouncy Particle Sampler (BPS), and the Randomised Hamiltonian Monte Carlo (RHMC), in terms of their jump rates and kernels, this paper shows how to obtain tractable closed-form and approximations for the jump rates and kernels of the time reversed PDMPs. Theoretically, this paper then proposes a total variation bound between the “learnt” data distribution and the true data distribution when the base distribution is a Gaussian. This is a useful property, quite similar to bounds that have been proposed before in the literature (for example, for the Ornstein-Uhlenbeck process in [1, Theorem 5.2.]). Finally, the paper proposes two empirical techniques to learn the time reversals akin to score-matching. First, for the ZZP process, inspired by score-matching techniques, the authors propose a ratio-matching technique. Secondly, for the BPS and RHMC processes, the authors learn normalising flows for the time reversal. They then show promising results in low-dimensional and MNIST generative modelling applications as a proof-of-concept. [1] Dominique Bakry, Ivan Gentil, Michel Ledoux, et al. Analysis and geometry of Markov diffusion operators, volume 103. Springer, 2014.

Strengths

I quite like the structure and formulation of the paper. The main goals and approach is elucidated quite clearly, and the mathematical preliminaries, while dense, seem correct for me. In my understanding, it is a known fact that all PDMPs admit an equivalent time-reversal, but these are quite hard to calculate in general. In this paper, building on theory involving jump Markov Processes in [2], the authors derive expressions for time-reversal jumps and kernels for 3 different PDMPs. In general, deriving the backwards time reversal and then also designing an empirical scheme with a neural network architecture and loss function would be a substantial contribution, but this paper has many additional contributions on top of that. The numerical experiments seem compelling, even if a little small scale. However, this paper seems like a proof-of-concept on the use of time-reversed PDMPs for generative modelling, and I think the theoretical contributions along with the design of the loss functions and training paradigms are a pretty significant contribution already. [2] Giovanni Conforti and Christian Léonard. Time reversal of markov processes with jumps under a finite entropy condition. Stochastic Processes and their Applications, 144:85–124, 2022.

Weaknesses

Fundamentally, I think the paper lacks a convincing argument about why generative modelling with PDMPs would fundamentally be more useful than traditional generative modelling. I understand that there were some arguments made in the introduction of the paper, namely Lines 35-36 (“such as better scalability and reduced computational complexity in high-dimensional settings”). However, it is really unclear to me how this argument actually translates to the empirical score-matching (or normalising flow training) objectives that the authors formulate, vs an approach like DDPM. The experimental section is quite lacking in details and comparisons about how the PDMP approach improves along any number of axes, beyond the qualitative plots. For example, I can think of many axes of improvement that could be discussed - sample efficiency (how many training datapoints are needed to learn the time-reversal given that the process is partly deterministic), mixing rates towards the Gaussian for their time reversal. Usually, SDEs such as the Ornstein-Uhlenbeck process are quite quick at mixing towards a Gaussian, making them quite nice to use when reversing a Gaussian distribution as the base. For partly deterministic processes, is this easier or harder to do? Are there any comparisons to regular Markov process methods that can show that having an irreversible Markov process is beneficial here? I believe that this is a big factor in why PDMPs are alluring, and their irreversibility makes them mix faster and use less data [3]. Any experiments showing sample efficiency and mixing rates would be really beneficial here. I am worried that there are many subtleties in the training and sampling procedures of diffusion models, and indeed there are many papers focusing solely on the empirical training tricks that can improve generative modelling, and comparing to a vanilla DDPM model doesn’t properly ablate the technique. I would be hesitant to rely on these empirical results as a surefire sign of improved modeling, which is frustrating, as theoretically, the paper does seem to point to this being the case, and I really do want to believe. Furthermore, I think the paper could benefit from being a lot clearer about the specific advantages of PDMPs vs other stochastic processes for generative modeling. This is barely mentioned, but does form the crux of the empirical results. This made it difficult for me to read through the theoretical developments, proofs and theorems without knowing the reason why we would really want to do this in the first place. I also think the paper can also benefit from being more explicit in how their developed score matching and normalizing flow training differs from traditional methods (maybe an algorithm block), as this would be something really interesting to practitioners looking to adopt existing codebases to using PDMPs instead. [3] Bierkens, J., Fearnhead, P., and Roberts, G. (2019), “The zig-zag Process and Super-Efficient Sampling for Bayesian Analysis of Big Data,” The Annals of Statistics, 47, 1288–1320. DOI: 10.1214/18-AOS1715

Questions

I’ve summarised a few of my questions in the Weaknesses section, but I list a few more questions here I would love some clarity on - You mention that [2] originally suggested time reversal theorems for jump Markov processes. I found it really difficult to figure out what the exact distinctions are between the theoretical formulations of [2], and the specific novelty of this paper. I’d love clarifications on what the additional mathematical frameworks introduced are, that adopt this framework to PDMPs. I would really like a better discussion on Equation 9, talking about what the different components of the inequality correspond to, and if there are any insights to gain about why PDMPs perform better or differently to something traditional like an OU process. In Line 238-239, you mention that it’s easy to subsample the loss across dimensions. Is this still an unbiased, consistent estimator for the continuous time loss? You mention that PDMPs typically have several types of jump rates and kernels, and provide Equation 2 as a parametric family of possible jumps and kernels. Is this the only possibility for constructing jumps and kernels? At the end of each section describing one of the 3 PDMPs, the authors mention that there has been work showing exponential convergence to invariant distributions for all the PDMPs. How does this exponential convergence compare to the OU process? Is there actually faster convergence due to the inherent irreversibility of PDMPs?

Rating

7

Confidence

4

Soundness

3

Presentation

2

Contribution

4

Limitations

Yes, they have addressed any potential limitations.

Response to authors' rebuttal

Thank you to the authors for providing a comprehensive rebuttal. I am satisfied that the authors have addressed my questions and I thank them for the incorporating my feedback into their revised paper.