PCF-GAN: generating sequential data via the characteristic function of measures on the path space

Summary

This paper proposes PCF-GAN, which aims to improve the effectiveness of the discriminator in differentiating the time series distributions by utilizing the path characteristic function (PCF) as a principled representation of the time series distribution within the discriminator. The authors also give the theoretical foundation of the PCF distance, proving its properties that ensure stable and feasible training of PCF-GAN. Extensive experiments show the effectiveness of the proposed method.

Strengths

The newly proposed metric in the discriminator for handling sequential data is interesting. Also, the author provides theoretical evidence for the analytical properties of the proposed loss metric.

Weaknesses

- The presentation of this work is kind of confusing, making it difficult to fully comprehend the content. Specifically, in Figure 3, it is unclear what the generator loss represents if it is computed on the embedding vector. Additionally, it is unclear why the generator and regularization loss can enhance the discriminative power in Equation 10. Clarifying these help audience better understand. - The related work section consists of only one paragraph, which may overlook some relevant research in the field. - The ablation study of every component is required to demonstrate the effectiveness individually. - Regarding the sequential data, I am wondering how does this method work well on more challenging domains like video data since videos are also sequential in nature.

Questions

Please see Weaknesses

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

Please see Weaknesses

Summary

The paper looks at generative models for times series data. It proposes a new GAN method, based on a novel discriminator. The path characteristic function (PFC) is used as a representation of the time series distribution. Using this, a distance between two distributions is defined (PCFD) as well as a way to approximate it (EPCFD). Afterwards, it is shown how EPCFD can be used as a discriminator in a GAN scenario (resulting in the PCF-GAN). To scale it to larger input dimensions, the authors follow previous literature and introduce a parameterised dimensionality reduction of the inputs and introduce different losses in an effort to ensure that it behaves as expected. Finally, they conduct a number of experiments, showing that PCF-GAN compares favourably to competitive time-series GANs.

Strengths

The method and its different components was well motivated and presented. The paper presents a compelling theoretical motivation for the method, giving evidence on why it might be preferable to alternatives s.a. WGAN. The experimental results are well analysed and presented.

Weaknesses

The reconstruction functionality is not motivated, nor explained. It can be unclear to the unfamiliar reader why one would want it, given that we can just generate images. Related work - similar work is only lightly touched upon and the advantages/disadvantages are not outlined, making it harder to compare. The loss function in Eq. 9 is not arrived at in a principled way and has two hyper-parameters which might be difficult to tune? In the experiments, the dimensionality of the inputs, as well as the length of the sequences appears rather low. This suggests potential scalability issues?

Questions

In Eq. 9, do you expect training only on “ -L_recovery - L_regularisation” to perform much worse?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The method has computational limitations which limits the number of dimensions which it can be applied to. As mentioned before, an embedding functions is trained to reduce the dimensionality of the inputs, however, if the effective dimensionality is high, the method would fail.

Summary

The paper proposes an approach to improve time-series modeling in Generative Adversarial Network framework by using path characteristic function as the embedding of the time series sample. It explains the feasibility of PCF distance and how to integrates it as a distance measure between two time series data in order to help the discriminator learn better generative features. The goal and benefit of the proposed method is to empirically improve the generation quality of time series data.

Strengths

The paper explores using a novel loss function in Generative Adversarial Network framework. It computes the proposed empirical path characteristic function distance between generated and given sample to learn better time series generative features. The proposed method outperforms other generative baselines on the given datasets. The paper is well written ( and includes the codebase in appendix for reproducibility).

Weaknesses

It would be great to have to following experiments in order to compare fairly with the existing baselines: 1. Comparison with existing baselines like COT-GAN on metrics like (a) the sum of the absolute difference of the correlation coefficients between channels avg over time (b) absolute difference between the correlation coefficients of real and generated samples. 2. Comparison on high dimensional dataset (like Sprites and human action sequences with FID, KID scores )

Questions

N/A

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

As mentioned in the weakness section, having more ablations and experiments for comparison would help the reader better understand the efficacy of the proposed approach.

Summary

This paper proposes a path characteristic function GAN, named PCF-GAN, for learning to generate time-series data. More specifically, the authors mainly employ the rough path theory to build the PCF distance, such that the temporal cues can be encoded by unitary features, enabling the PCF to learn sequential data. The PCF distance is then proved to be complete and favorable for minimizing the difference between two stochastic processes. Even though, an encoder-decoder-based embedding is proposed to compare two random processes in the latent space of reduced dimensions, catering for the ease of training in practice. Experimental results have also verified the effectiveness of the proposed PCF-GAN.

Strengths

The established theory on PCF is novel and complete, by far my understanding. The authors also propose a practical way of minimizing the PCF distance, by using an encoder-like embedding that is able to reconstruct signals.

Weaknesses

1. I am appreciated the theory behind the PCF. However, the usage of path theory needs to be clarified, when aiming at improving characteristic functions to represent stochastic process. The authors claim that CF-metric fails to capture temporal dependency of sequential data, which is true. However, why using path theory on characteristic functions can well address the temporal dependency, given the fact that there are other representations of the characteristic function of a stochastic integral? 2. The authors claim that the embedding operation does not have any gradient constraints. However, when adding calculating the EPCFD after the embedding layers, the continuous and differential properties may not hold, without the restriction on the embedding layers. I am not sure whether the reconstruction regularization can compensate strictly. The authors are encouraged to elaborate more on this. 3. Another weakness is regarding the experimental validations, which were performed based on low-dimensional time-series datasets. The PCF-GAN is shown to beat the COT-GAN on these datasets. However, COT-GAN is also able to generate short video clips, in which the scenarios are much complicated. Would the PCF-GAN scale well to the higher-dimensional generation tasks, such as short video generation? Also the comparing baselines are not the state-of-the-art. It is the fact that many recent methods have been reported to surpass COT-GAN and TimeGAN, for example, [1] and [2]. [1] Seyfi, Ali, Jean-Francois Rajotte, and Raymond Ng. "Generating multivariate time series with COmmon Source CoordInated GAN (COSCI-GAN)." Advances in Neural Information Processing Systems 35 (2022): 32777-32788. [2] Jarrett, Daniel, Ioana Bica, and Mihaela van der Schaar. "Time-series generation by contrastive imitation." Advances in Neural Information Processing Systems 34 (2021): 28968-28982.

Questions

1. In Equation (4), why x_0=I_m. Wouldn't it be y_0=I_m for the differential equation? 2. In Line 254, what does it mean by IGM? Should it be IPM? 3. Why for the regularization loss, this paper employs a different \theta'_M? How \theta_M and \theta'_M were implemented in this paper?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

Please see my weakness.

Summary

The paper presents a new metric for the distributions on the path space via PCF and provides theoretical proofs for analytic properties of the proposed loss metric which benefits GAN training. It introduces a novel PCF-GAN to generate & reconstruct time series simultaneously. It compares the proposed method with prior work and shows improved performance.

Strengths

- The paper is relatively well-written and the proposed approach is described clearly. - The authors provide theoretical grounding for their work. They prove PCF's characteristicity, boundedness, differentiability with respect to generator parameters, and its weak continuity which ensures the stability and feasibility of training the PCF-GAN. - Experimental results demonstrate improved performance compared to several prior works. The authors provide comparisons with TimeGAN, CotGAN and RGAN on RV, Stock, Air and EEG datasets using discriminative and predictive metrics.

Weaknesses

- There are concerns regarding missing comparisons with related work. [A, B] also propose generative models for time series generation. [A] combines the adversarial training of GANs and the exact maximum likelihood training of CTFPs into a single framework. It designs an invertible generator, and adopt an autoencoder, on whose hidden space of the GAN performs the adversarial training. [B] uses Deep Euler representation and Wasserstein distances to propose three generative methods for times series. Two generative method EWGAN and EDGAN demonstrate an accuracy similar to state-of-the-art GAN generators and show better performance for capturing temporal dynamic metrics of the time series. The third method CEGEN is based on a loss metric computed on the conditional distributions of the time series. These papers are not mentioned and no comparisons are provided with them. - There are metrics such as FID in prior work, e.g. [B], which are not used in the experiments. [A] GT-GAN: General Purpose Time Series Synthesis with Generative Adversarial Networks; Jeon et al.; [B] Conditional Loss and Deep Euler Scheme for Time Series Generation; Remlinger et al.;

Questions

The authors need to clearly distinguish their work with existing literature and provide comparisons with prior work (e.g. [A, B]).

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors have adequately discussed limitations of their work.