BACKTIME: Backdoor Attacks on Multivariate Time Series Forecasting

Multivariate Time Series (MTS) forecasting is a fundamental task with numerous real-world applications, such as transportation, climate, and epidemiology. While a myriad of powerful deep learning models have been developed for this task, few works have explored the robustness of MTS forecasting models to malicious attacks, which is crucial for their trustworthy employment in high-stake scenarios. To address this gap, we dive deep into the backdoor attacks on MTS forecasting models and propose an effective attack method named BackTime.By subtly injecting a few stealthy triggers into the MTS data, BackTime can alter the predictions of the forecasting model according to the attacker's intent. Specifically, BackTime first identifies vulnerable timestamps in the data for poisoning, and then adaptively synthesizes stealthy and effective triggers by solving a bi-level optimization problem with a GNN-based trigger generator. Extensive experiments across multiple datasets and state-of-the-art MTS forecasting models demonstrate the effectiveness, versatility, and stealthiness of \method{} attacks. The code is available at \url{https://github.com/xiaolin-cs/BackTime}.

Paper

Similar papers

Peer review

Reviewer Bpy47/10 · confidence 2/52024-06-29

Summary

The paper titled "BACKTIME: Backdoor Attacks on Multivariate Time Series Forecasting" introduces a novel method called BACKTIME, aimed at exploring the robustness of multivariate time series (MTS) forecasting models against backdoor attacks. BACKTIME enables an attacker to subtly manipulate predictions by injecting stealthy triggers into the MTS data. Through a bi-level optimization process using a graph neural network (GNN)-based trigger generator, the method identifies vulnerable timestamps and crafts effective, covert triggers. Comprehensive experiments across various datasets and cutting-edge MTS forecasting models demonstrate the efficacy, flexibility, and stealth of BACKTIME attacks.

Strengths

- The BACKTIME method offers a fresh perspective on backdoor attacks in MTS forecasting, demonstrating the feasibility and impact of such attacks. - The experimental setup is comprehensive and includes multiple datasets and state-of-the-art forecasting models, showcasing the method's versatility. - The paper provides clear explanations of the bi-level optimization process and the role of the GNN-based trigger generator, facilitating understanding of the attack mechanism. - The research highlights the need for robustness in MTS forecasting models, particularly in high-stake scenarios where malicious attacks could have serious consequences.

Weaknesses

- The paper does not adequately address potential limitations or discuss the robustness of the method against countermeasures. A more detailed analysis of the limitations and how they might affect the practical applicability of BACKTIME would strengthen the paper. - The potential negative societal impacts of such attacks are not sufficiently explored. Given the sensitive nature of MTS forecasting in areas like climate, epidemiology, and finance, a deeper discussion on the broader implications is warranted.

Questions

- What are the potential limitations of the proposed method, especially regarding its generalizability to other types of time series data or forecasting models?

Rating

7

Confidence

2

Soundness

3

Presentation

3

Contribution

3

Limitations

- The authors have not adequately addressed the limitations of their work, nor have they discussed the potential negative societal impacts of BACKTIME.

Reviewer Bpy42024-08-13

Thank you for the detailed rebuttal. The clarification of BACKTIME's limitations and the discussion on societal impacts are appreciated. Addressing these aspects will indeed strengthen the paper. Please ensure these points are integrated into the manuscript for the next review. Continue to consider potential defenses and broader implications for various applications. My concern has been addressed. I will raise my score to accept.

Authorsrebuttal2024-08-13

Response to Reviewer Bpy4

We want to express our sincere thanks for your recognition of our work. Answering your insightful comments helps us improve the quality of our work. In the revised version of our paper, we promise to include discussions on potential defenses and broader implications. We are confident that these additions will enrich our contribution and provide a more comprehensive understanding of the impact and context of our work.

Reviewer PmgC8/10 · confidence 4/52024-07-10

Summary

This paper proposed a backdoor attach method for multivariate time series forecasting, where only a few works focus on this topic. The injected stealthy triggers are interesting and effective in destroying raw data.

Strengths

The originality of this paper is solid, since this is the first work to consider the backdoor attack in MTS forecasting. The quality and clarity of this paper are also good for understanding. The significance of this paper is relatively well-clarified.

Weaknesses

1. The Trigger Generation and Bi-level Optimization are not clearly presented. Please extend the description of trigger generation and effectiveness. 2. The experiments need to be expanded, and the advantages of the proposed method are not well validated. 3. Numerically, the authors could consider comparing their method with more baselines. There are some studies on backdoor attacks for forecasting models.

Questions

1. The Trigger Generation and Bi-level Optimization are not clearly presented. Please extend the description of trigger generation and effectiveness. 2. The experiments need to be expanded, and the advantages of the proposed method are not well validated. 3. Numerically, the authors could consider comparing their method with more baselines. There are some studies on backdoor attacks for forecasting models.

Rating

8

Confidence

4

Soundness

4

Presentation

3

Contribution

4

Limitations

1. What is the technical drawback of the proposed method? E.g., attack efficiency and hidden effect. 2. Are there any initial ideas to be able to look forward from a defense perspective?

Reviewer mt9q6/10 · confidence 2/52024-07-14

Summary

This paper primarily discusses how to conduct a backdoor attack on the multivariate time series forecasting task, proposing a two-stage training approach. The core idea is to identify timestamps with significant differences in MAE of the Clean model and Poisoned model on poisoned data points. Simultaneously, the trigger generation function is learned by minimizing the MAE loss of the model's prediction of the target pattern with triggers as input.

Strengths

1. This is a quite interesting and important research topic. However, the example provided by the author regarding traffic prediction is not convincing enough. Traffic prediction data is collected and trained by private systems. If one can already access the system to inject toxic data, there would be simpler and more effective methods than backdoor attacks. If the author can provide some scenarios where the time series data points are provided by third-party systems, it would better illustrate the necessity of this paper. 2. The author's writing is clear and easy to follow. 3. The author has proposed a framework that can be applied to various time series prediction backbones and has conducted a comprehensive evaluation of effectiveness.

Weaknesses

1. The baseline methods in the paper are heuristic, and the worse MAE of these methods compared to the proposed method in the paper does not effectively illustrate the issue. 2. The types of anomalies illustrated in the paper are relatively simple.

Questions

1. In Equation 3, according to the paragraph from lines 189-190, the clean loss should be the MAE using the clean samples to predict the ground truth, and the attack loss should be the MAE using the poisoned samples to predict the target pattern, right? However, in Equation 3, both MAE losses target $X^{ATK}_{ti,f}$. Here, clarification from the author is requested. 2. The training of the trigger generation network depends on initially manually generating some attack samples. Does the quality of these initial samples significantly affect the following training process? 3. The two-stage training is very unstable, which the author mentioned in the paper, but there is no detailed description in the text. It would be helpful to understand the rationality of the model design if the author could provide the training loss change curve.

Rating

6

Confidence

2

Soundness

2

Presentation

4

Contribution

3

Limitations

The author has discussed in the paper how to prevent the injected data points from having too large a magnitude and how to address training instability issues.

Reviewer mt9q2024-08-11

Thank the author for the reply. The explanation regarding clean loss and trigger generation has addressed my concerns. However, the examples in response to reviewer Bpy4 has not convinced me. In both the medical and stock market, the attacks are still conducted after data collection. If one can access the data in such private system, the most effective method would be to directly tamper with the data rather than using the indirect approach of constructing toxic samples to influence the model. The current method in the paper requires training to determine what kind of attack samples to generate, which I believe is a limited contribution. I am more looking forward to seeing attack methods and defense methods that are training-free. In total, I think this paper has not yet reached the level of a score 7 (clear accept), and I maintain my current score of 6 as weak accept."

Authorsrebuttal2024-08-12

Response to Reviewer mt9q

We are very glad to know that your concerns about the clean loss and trigger generation have been addressed. We also want to thank you for the follow-up questions and are happy to share more answers. First, we would like to point out that standard backdoor attacks belong to the category of **data-poisoning** attacks, where modifications/augmentations to training data are a necessary and indispensable requirement in backdoor attacks across various domains [1,2,3,4], such that the attacked model learn the strong association between triggers and target pattern (or target class) in the training data, thereby altering its predictive behavior. Unlike traditional attacks that might rely solely on exposed data and could be short-sighted or ineffective, backdoor attacks strategically manipulate the training data to embed malicious behavior, which remains dormant during normal operations but can be triggered under specific conditions. This approach ensures that the attack is both stealthy and persistent. Furthermore, under this data-poisoning setting, backdoor attacks can be highly threatening because neural network training requires a large amount of data, which is difficult to accomplish by only using private data. Therefore, by stealthily poisoning public datasets, attackers can pose a great threat to neural networks' wide application. For example, in the healthcare field, it is hard for a single hospital to collect enough data to train a robust model. Hence, hospitals are likely to delegate to AI-related agencies, and those agencies mix the hospital-provided data with large-scale public data (e.g., scraped from the internet) to enhance model performance. By poisoning widely used public datasets, such as ERP [5] and OpenNeuro [6] datasets, backdoor attacks could pose a significant threat to the medical control systems used by hospitals. The suggestion of a training-free strategy is quite interesting but indeed presents significant challenges to attack efficiency based on the current developments in the research community. Since a training-free backdoor attack, neither has any knowledge of the architecture of models to be trained nor has access to the data during training, the trigger can only be manually set. However, time series data varies greatly in both frequency and amplitude, ranging from traffic data to weather patterns. It is exceedingly difficult for a manually designed trigger to perform an effective attack across different time series datasets. Considering this, we believe that developing a training-free backdoor attack is a highly challenging research problem, and we would be very interested in exploring this in future research. [1] Badnets: Identifying vulnerabilities in the machine learning model supply chain. [2] Backdoor attack with imperceptible input and latent modification [3] Textual backdoor attack for the text classification system. [4] A backdoor attack against lstm-based text classification systems. [5] https://erpinfo.org/erp-core [6] https://openneuro.org/

Reviewer PmgC2024-08-12

Thanks for the authors' efforts in the detailed response. My concern has been addressed. I will raise my score to strong accept.

Authorsrebuttal2024-08-13

Response to Reviewer PmgC

We sincerely thank you for your appreciation of our work. Your comments have been invaluable in guiding us in improving the quality of our research. In the revised version of the paper, we promise to include detailed descriptions of the trigger generation and bi-level optimization processes, as requested. Additionally, we will incorporate a discussion of potential defenses against the proposed attack. We believe that these enhancements will significantly strengthen our work and provide a more comprehensive understanding of its contributions.

Program Chairsdecision2024-09-25

Decision

Accept (spotlight)

© 2026 NYSGPT2525 LLC