PureGen: Universal Data Purification for Train-Time Poison Defense via Generative Model Dynamics

Summary

The paper presents a novel approach to securing model's performance (robust and natural accuracy) against train-time data poisoning attacks by introducing a set of data purification transformations during training, specifically employing Energy-Based Models (EBM) and Denoising Diffusion Probabilistic Models (DDPM). The effectiveness of the proposed method is demonstrated through evaluations on CIFAR-10, Tiny-ImageNet, and CINIC-10 datasets.

Strengths

1. The paper is generally well-written and structured. However, certain sections could benefit from additional clarity and elaboration to enhance the overall presentation quality. 2. The elucidation of the L2 Distance in Section 3.4 is commendable, offering a lucid explanation of the method's underlying mechanism for aligning poisoned data with the benign data distribution. 3. The exploration of different variants of the proposed method in Section 4.4 is a valuable addition to the paper. 4. The method achieves state-of-the-art performance in the conducted experiments, with a comprehensive exploration of its capabilities across various scenarios.

Weaknesses

1. Section 3.1 could be improved by providing a more in-depth discussion on the application of EBM models within the context of the proposed method, rather than focusing predominantly on foundational concepts. 2. The relevance of Section 3.3 is questionable, as the equations introduced (e.g., Eq. 6) do not appear to be integral to the subsequent discussion. 3. The paper may benefit from a clearer articulation of its novelty within the field.

Questions

1. The application of EBM and DDPM for adversarial purification is gaining traction, particularly in defense against inference attacks [1]. Is this the first application of these models for training-time data poisoning defense? How does this work distinguish itself within the existing body of literature regarding its novelty? 2. The claim on Page 2, Line 45, that purification models require training with a POOD dataset may not present a significant challenge. Pre-trained DDPM models are readily available in open-source repositories and can be employed for data purification, as demonstrated in [1]. Could the authors conduct a comparative analysis between the performance of publicly available DDPM models and those trained in-house? [1] "(Certified!!) Adversarial Robustness for Free!." Carlini, Nicholas, et al. The Eleventh International Conference on Learning Representations. 2022.

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

See weakness and questions.

Summary

This paper studies the generative purification methods, i.e., EBM and DDPM-based purifications, as defenses against a set of data poisoning attacks.

Strengths

1. They suggest that a proper range of implementation steps in the EBM and DDPM-based purification methods matters for defense performance. 2. They conduct comprehensive experiments to evaluate the effectiveness of PureGen methods. Experimental results show that proposed defenses perform better than the baselines in the paper. 3. They study some considerations in practical scenarios including (a) training distribution shift in the generative model training; (b) training the generative model on poisoned data; (c) network architecture transferability 4. They explore some PureGen variants.

Weaknesses

1. The idea of employing generative methods to purify imperceptive noises has been explored, i.e. EBM [1] and DDPM [2]. As for training-time poisoning attacks, [3, 4] propose to use diffusion process to defend against availability attacks. Though this paper studies different data poisoning attacks such as Bullseye Polytope, Gradient Matching, and Narcissus, I don't think there is any essential difference in the defense mechanism. 2. Since the technical modification in DDPM lies in the truncation of training steps, it's beneficial to move Figure 4 to the main paper which illustrates how the the selection of steps influences the defense performance. [1] Yoon J, Hwang S J, Lee J. Adversarial purification with score-based generative models [2] Nie W, Guo B, Huang Y, et al. Diffusion Models for Adversarial Purification [3] Jiang W, Diao Y, Wang H, et al. Unlearnable examples give a false sense of security: Piercing through unexploitable data with learnable examples [4] Dolatabadi H M, Erfani S, Leckie C. The devil’s advocate: Shattering the illusion of unexploitable data using diffusion models

Questions

1. Can you provide more intuition about why sacrificing generative capabilities can improve poison defense? 2. Can AVATAR[1] work as a defense baseline against Bullseye Polytope, Gradient Matching, and Narcissus? 3. In Table 1, why are there no gradient matching results on CINIC-10 and no Narcissus results on Tiny-ImageNet? 4. Why do some Poison Success cells show no standard deviations in Table 1,2? And is the bold font abused in these tables? [1] Dolatabadi H M, Erfani S, Leckie C. The devil’s advocate: Shattering the illusion of unexploitable data using diffusion models

Rating

4

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

See weaknesses and questions.

Summary

This work introduces PureGen, a method to purify a potentially poisoned dataset before the dataset is used to train a classifier. The work explores using Langevin MCMC with both EBMs and diffusion models to remove potential adversarial artifacts from data, with the justification that MCMC sampling should move a sample away from the adversarial artifacts and towards the data distribution. The number of Langevin steps is chosen to be enough to remove adversarial artifacts while preserving the original image. Experimental comparison shows strong performance of the proposed method for training robust classifier with a variety of data poisons compared to existing approaches.

Strengths

* The presentation of the method is clear and straightforward, and the paper is easy to follow. * The work proposes interesting adaptions of Langevin purification for white-box adversarial defense to the domain of data poisons. * Empirical results show that the proposed defense can outperform existing defense. There is a thorough examination of different poisons and comparison with existing methods.

Weaknesses

* The defense requires costly Langevin iterations and has a higher computational burden compared to existing methods. I appreciate that this is acknowledged clearly and discussed by the authors in the limitations section. * My main concern about this work involves the situation where the data poisoner is aware of the defense method that will be applied to the dataset. Developing attacks that can adapt to different defense strategies is standard in the white-box attack literature, although I am not as familiar with such procedures in the poison literature. Will the defense remain robust if the attacker is aware of the defense strategy? See, for example, [a]. [a] https://arxiv.org/abs/1802.00420

Questions

Could the authors discuss the situation where the attacker is aware of the defense? I am willing to raise my score if this concern can be addressed.

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

Limitations are adequately discussed.

Summary

This paper proposes a stochastic preprocessing defense technique, named PureGen, against train-time poisoning attacks, with EBM-Guided and Diffusion-Guided sampling processes. First, with EBM-based purification, called PureGen-EBM, the purifier first evaluates the (unnormalized) energy function of the images. Then, this considers the high-energy images as the 'posioned' images, and purifies with the stochastic preprocessor $\Psi_T$. On the other hand, with Diffusion-based purification, called PureGen-DDPM, the purifier first add Gaussian noises to the inputs and run the reverse diffusion process using DDPM.

Strengths

* This paper demonstrates purification results with respect to diverse poisoning attacks and data availability attacks, which provides a new benchmarks on the adversarial learning communities. * To the best of my knowledge, this is the first result to implement adversarial purification to poisoning attacks, achieving superior performances compared to other existing methods.

Weaknesses

* The idea of using adversarial purification to adversarial attacks is already well-known, and this is an increment of the adversarial purification to other kinds of attacks. * There are still scenarios that also breaks the adversarial robustness of the purification models such as BPDA+EOT [Athalye et al, 2018]. To mitigate this, additional consideration like fine-tuning on the adversarial perturbation models is required. [Lin et al., 2024]. Without this, the purified images will be easily poisoned with stronger poisoning attacks, which also involves with the purifier in the poisoning steps. * Some additional validation on using the diffusion models on adversarial purification should be addressed: see __Questions__. * In my opinion, with improved sampling methods using a higher-order solver of consistency-based distillation models, the purification can be drastically faster compared to both the proposed methods and the existing method. I presume that this can be easily improved: See __Questions__. [Athalye et al, 2018] Obfuscated Gradients Give a False Sense of Security, ICML 2018 \ [Lin et al., 2024] Adversarial Training on Purification (AToP): Advancing Both Robustness and Generalization, ICLR 2024

Questions

* In the PureGen-DDPM process, the dataset is trained with DDPM fewer steps, (250 steps rather than the conventional 1000 steps). Nevertheless, training DDPM with fewer discrete steps yields heavier posterior mismatch between the DDPM posterior covariance and the optimal covariance, according to [Bao et al, 2022]. I doubt that the purification gain from less timesteps from Figure 4 (bottom right) is just a side-effect of this posterior mismatch, which is not rigorously intended. I will be more convinced if some explanation on setting the fewer steps in PureGen-DDPM for training. * Even though the time complexity of the proposed purification method is comparable to the existing methods, this can be easily improved with some higher-order solvers without any additional training. The paper will be much stronger if more results with some solvers are addressed. [Bao et al, 2022] Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models, ICLR 2022 --- Minor typos * (Appendix C) Intition $\to$ Intuition

Rating

4

Confidence

5

Soundness

3

Presentation

3

Contribution

1

Limitations

N/A

Thanks for the response. I will raise my score.

I appreciate the efforts to investigate the defense-aware data poison, and it is reassuring to see the defense retains security. This addresses my main concern. While the EBM and diffusion purification methods fall within established practice, there appears to be a consensus among reviewers that this work does cover an area that has not yet been directly investigated in published work, namely purification-type defense applied for train-time data poisoning/purification. It does seem like there should be a work like this as a reference point for the community and the authors performed a wide array of experiments to investigate this scenario. I understand the concerns of other reviewers regarding novelty, but I nonetheless feel this work makes an expected but useful contribution. I will increase my score since my main concerns have been addressed.