Finding Order in Chaos: A Novel Data Augmentation Method for Time Series in Contrastive Learning

Summary

This paper presents a new time series augmentation method which adopts Mixup on the amplitude and phase, respectively, after transforming the time series in the frequency domain, thus avoiding the destructive extrapolation from linear Mixup.

Strengths

1. The authors propose a novel mixup method in the frequency domain for time series, avoiding the destructive extrapolation resulting from linear Mixup. 2. The theoretical proof is complete and has some reference value.

Weaknesses

1. It is unclear whether it works when considering amplitude or phase alone for Mixup. The relevant ablation experiments must be performed consistently with the parameter settings of the original method. 2. Lack of convincing examples. For example, to illustrate the problem, perform Mixup on two samples where the frequency domain information in both samples that are strongly correlated with I(x,y) exactly cancels each other and shows whether the newly generated samples have specific frequency domain information that can be identified. 3. The paper's content does not explain how to "Find Order in Chaos" mentioned in the title and how the "chaos" is reflected. And I'm confused about how the so-called "control the degree of chaos" proposed by the authors is done. 4. Assumption 2.1 is too idealistic. The label information of time series is not only related to the time domain but also the frequency domain, and focusing on the frequency domain information only does not achieve the classification well.

Questions

1. The ablation experiments considering amplitude or phase alone for mixup need to be performed consistently with the parameter settings of the original method. 2. Please give practical examples of how Mixup may corrupt the frequency domain information used to discriminate the generated samples. 3. It is not clear how the so-called "control the degree of chaos" proposed by the authors is done. 4. For data augmentation of time series, why does this paper choose to perform augmentation in the frequency domain instead of the time domain? In general, it is difficult to use only frequency domain information for classification over the time domain for time series classification tasks [1]. In contrast, the combination of time and frequency domain information can effectively improve the model's classification performance [2]. 5. What are the advantages of performing data augmentation in the frequency domain instead of doing data augmentation in the time domain of time series data? 6. Why are eight datasets chosen for experiments in this paper? Its inconsistency with existing benchmark datasets for time series prediction [3,4,5], classification [6,7] and anomaly detection [8] tasks. In addition, the comparison methods in this paper do not include benchmark methods in the time series domain. For example, benchmark methods for time series prediction include Informer [3], CoST [4], and FEDFormer [5], etc., time series classification methods include OS-CNN [6] and DSN [7], etc., and time series anomaly detection includes Anomaly Transformer [8], etc. [1] Cross reconstruction transformer for self-supervised time series representation learning. arXiv, 2022. [2] Self-Supervised Contrastive Pre-Training for Time Series via Time-Frequency Consistency. NeurIPS, 2022. [3] Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting. AAAI, 2021. [4] CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting. ICLR, 2022. [5] FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting. ICML, 2022. [6] Omni-Scale CNNs: a simple and effective kernel size configuration for time series classification. ICLR, 2022. [7] Dynamic Sparse Network for Time Series Classification: Learning What to "See". NeurIPS, 2022. [8] Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy. ICLR, 2022.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

The authors did not mention the limitations. I do not have any comments on this point.

Summary

This paper proposes a novel mixup method for non-stationary time-series data to generate positive samples for the contrastive learning formulation. The proposed method mixes the magnitude and phase of each frequency component of two samples that are close in the latent space of a variational autoencoder. The authors prove that the mixup process in the frequency domain does not cause the loss of information while it generates diverse samples. The paper provides an extensive empirical evaluation to show that the proposed method learns better representations than existing contrastive learning baselines on eight time series datasets from three tasks.

Strengths

The paper studies an important and under-explored problem. Although contrastive learning is proven to work well on images, its performance on time series is yet limited due to the lack of good time series augmentations. While many existing work tries to find the good time series augmentations through extensive empirical studies, this paper proposes a theoretically well-grounded mix-up method for time series augmentation. Moreover, the empirical evaluation is also extensive and convincing.

Weaknesses

1. The hyperlinks of the citations and the references are missed. 2. I don't find where $x^*$ is defined. 3. It would be great if there is an algorithm box explaining the whole process of drawing samples, computing the degree of the augmentation, performing the mixup, and then training.

Questions

1. How do you select $\beta$? Would a very large (close to 1) $\beta$ limit the diversity of the augmented samples? 2. What does the instance selection (line 268) mean exactly?

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

3 good

Contribution

4 excellent

Limitations

Yes

Summary

This paper is about a novel data augmentation method that can be used in contrastive learning for time-series tasks and aims to connect intra-class samples together, and find order in the latent space. The proposed method builds upon the mixup data augmentation technique by controlling the degree of chaos created by data augmentation. More specifically, the augmentation method considers the phase and the amplitude information as two separate features and then generates positive samples by controlling the mixup coefficients for each feature for each randomly chosen pair. This process helps to generate features that enhance intra-class similarity and help contrastive learning to learn class separated representations. The proposed method is evaluated using three time-series tasks: heart rate estimation, human activity recognition and cardiovascular disease detection and against state of the art comparison methods.

Strengths

- The paper is about an interesting topic, contrastive learning in time-series. - The proposed methodology is based on the idea of controlling the degree of chaos created by data augmentation methods. These ideas can be applied in the augmentation part of contrastive learning for other domain as well. - The proposed methodology shows improvement in the performance over ten comparison methods on three tasks and eight datasets. Some time-series contrastive learning methods are missing from the comparison though.

Weaknesses

- The writing in the paper can be improved. The Method and Results and Discussion sections need some organization, adding subsections will help with the structure. - The proposed method has incremental novelty. This method builds on top of known components. The writing needs improvements to make the contributions of this paper more clear. - The experimental setup also seems to be missing some important state of the art. No time-series specific contrastive learning method is included in the experiments for comparison. - The proposed methodology seems to have similarities with TS-TCC [80] and TFC [22] (the reference numbers align with the ones in the main paper) and since these are state of the art in contrastive learning for time-series, they should be part of the comparison methods. - A comparison of the methodology of the proposed method with [22] and [80] is needed as these paper are around the same ideas. What is different in the contributions of the proposed method? - Dataset statistics are missing from the Results and Discussion section. Adding them in a table helps with the understanding of the results.

Questions

My main concerns are regarding the readability of the paper and improvements in the structure of the paper, and the experimental setup that is missing time-series contrastive learning models. The results and the discussion will be more convincing and the performance difference more important if the author include the results of some other methods, like [22] and [80]. A comparison of the methodology and the concepts of the proposed method with [22] and [80] is needed as these paper are around the same ideas. What is different in the contributions of the proposed method?

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

There are no limitations or negative societal impact from this work. The datasets used in the paper are all publicly available.

Summary

This paper explores data augmentation for contrastive learning that better aligns with the nature of time series. Specifically, they tailor the mix-up method separately to amplitude and phase in the frequency domain. A theoretical analysis illustrates how the method enhances task-specific information, contrasting it to the linear mix-up method which potentially discards task-specific information. Experiments on time-series from three domains also demonstrated the advantages compared with previous mix-up and contrastive learning methods.

Strengths

1. This paper studies an important problem of designing augmentation for contrastive learning that adapts to the non-stationarity of time series. 2. The paper presents theoretical analysis of the benefits of proposed mix-up method compared with vanilla linear mix-up method. 3. The paper shows better performance on time-series datasets of multiple domains.

Weaknesses

**Lack of baseline comparisons**: The paper compares with a list of mix-up methods and contrastive learning methods, and the authors also incorporate time-series augmentation methods into the framework in appendix. However, it would also be interesting to have (1) some direct comparisons with existing time-series augmentation methods, e.g., [1, 2] (both have mix-up augmentation in the time or frequency domain). (2) Contrastive learning methods dedicated for time-series domain [3, 4, 5, 6] (I am curious why [4] is not one of the baselines in experiments?) [1] FrAug: Frequency Domain Augmentation for Time Series Forecasting [2] Towards Diverse and Coherent Augmentation for Time-Series Forecasting [3] CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting [4] Self-supervised contrastive pre-training for time series via time-frequency consistency [5] Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding [6] TS2Vec: Towards Universal Representation of Time Series **Experimental analysis**: Why linear mix-up method performs well in activity recognition task? Is there any difference in the setting of activity recognition compared with the other two tasks? **Writing**: 1. Consider defining notations of x, \tilde{x}, x^* before or in Proposition 2.3. 2. The title metions "order" and "chaos"; hence, a detailed explanation of these terms, as well as how the proposed method comprehensively accounts for the "non-stationarity" of time series, would be beneficial.

Questions

See weakness above.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

The paper did not explicitly discuss their limitations. Some comments and questions regarding limitations of the work are discussed in the weakness and question parts above.