Diffusion-Based Adversarial Sample Generation for Improved Stealthiness and Controllability

Neural networks are known to be susceptible to adversarial samples: small variations of natural examples crafted to deliberately mislead the models. While they can be easily generated using gradient-based techniques in digital and physical scenarios, they often differ greatly from the actual data distribution of natural images, resulting in a trade-off between strength and stealthiness. In this paper, we propose a novel framework dubbed Diffusion-Based Projected Gradient Descent (Diff-PGD) for generating realistic adversarial samples. By exploiting a gradient guided by a diffusion model, Diff-PGD ensures that adversarial samples remain close to the original data distribution while maintaining their effectiveness. Moreover, our framework can be easily customized for specific tasks such as digital attacks, physical-world attacks, and style-based attacks. Compared with existing methods for generating natural-style adversarial samples, our framework enables the separation of optimizing adversarial loss from other surrogate losses (e.g., content/smoothness/style loss), making it more stable and controllable. Finally, we demonstrate that the samples generated using Diff-PGD have better transferability and anti-purification power than traditional gradient-based methods. Code will be released in https://github.com/xavihart/Diff-PGD

Paper

References (55)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer PvL77/10 · confidence 4/52023-06-29

Summary

The paper proposes a framework called Diffusion-Based Projected Gradient Descent (Diff-PGD) for generating realistic and stealthy adversarial samples to deceive neural network models. Diff-PGD utilizes a diffusion model to guide the optimization process, ensuring that adversarial samples remain close to the original data distribution while maintaining their effectiveness. The framework offers customization for specific attack scenarios, such as digital attacks, physical-world attacks, and style-based attacks. It separates the optimization of the adversarial loss from other surrogate losses, leading to improved stability and controllability. The authors demonstrate that adversarial samples generated using Diff-PGD exhibit better transferability and anti-purification power compared to traditional gradient-based methods.

Strengths

## Originality (+) The paper introduces a novel framework, Diff-PGD, which combines the use of diffusion models with adversarial sample generation. This integration of diffusion models into the generation of adversarial samples is a unique approach that has not been explored extensively in prior research. ## Quality (+) The paper provides a thorough description of the proposed framework, Diff-PGD, and its underlying principles. ## Clarity (+) The paper is well-written and effectively communicates the key concepts and ideas. ## Significance (+) By introducing the Diff-PGD framework, the authors offer a solution that improves the stealthiness of adversarial samples while maintaining their effectiveness. (+) The provided experimental evaluations (quantitative and qualitative) are convincing.

Weaknesses

(-) The phenomenon of the vulnerability of DNNs/ViTs to AEs is well known. How does the proposed attack perform against hardened models, i.e. adversarially trained models? (-) Did the authors experiment with transformer-based models, such as ViT? I would be interested in the attack performance of these models as well.

Questions

Please address my questions in the weakness section.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors elaborated on the limitations of the approach.

Reviewer B84v5/10 · confidence 4/52023-07-05

Summary

This paper proposes to use diffusion models to improve the quality of adversarial examples. It improves the Projected Gradient Descent (PGD) attack by performing a few steps of Stochastic Diffusion Editing (SDEdit) before taking each projected gradient descent (i.e., adversarial) step. The proposed method, DiffPGD, can also be adapted for regional adversarial and style-customized attacks. Experiment results show that DiffPGD can output high-quality images that attack classifiers effectively.

Strengths

1. The idea of using diffusion models to improve the quality of adversarial examples is interesting. The proposed method is shown to generate more natural-looking adversarial examples with better transferability. 2. The evaluation considers different attack scenarios, including digital and physical attacks.

Weaknesses

1. The proposed method is much more computationally expensive (in terms of both time and GPU memory) than regular adversarial attacks, while its advantage over existing methods seems marginal (Figure 6(a)). 2. Many of the experiment results only provide less than five examples, which are inadequate to support the claims (i.e., the four questions listed in Lines 220 - 224) about the proposed methods. For example, the result in Figure 6 (a) is conducted only on 250 samples (rather than the entire 50,000 ImageNet validation samples). The answers to Q2 and Q3 are based only on the examples shown in the figures. It is unclear how well Diff-PGD and its variants perform at a larger scale. 3. The comparison with baselines is not very comprehensive. For example, Figure 4 only shows a qualitative comparison with AdvCam, but no attack success rate is considered in this comparison. While DiffPGD is shown to produce more realistic-looking images in Figure 4, it is not clear if they are as effective as baseline adversarial examples.

Questions

1. The evaluation metric used in Table 2 is unclear. Is it the attack success rate?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

1. As I mentioned in the weaknesses, the major limitation of this work is its computation overhead. It seems impractical to incorporate this attack method into model training as a defense method, while the original PGD can be easily applied to adversarial training. 2. Limited baselines are considered in the current experiment section, which makes the effectiveness of the proposed method unclear. For example, when demonstrating the transferability and anti-purification of Diff-PGD, this paper only compares to PGD. It is important to compare Diff-PGD with more GAN-based adversarial attack methods to show its advantage.

Reviewer x4rZ5/10 · confidence 4/52023-07-06

Summary

This paper propose a gradient-based adversarial attack method that use diffusion model to generate adversarial examples.The proposed method can generate adversarial examples with high stealthiness and controllability.

Strengths

1.The proposed method can improve the stealthiness, anti-purification ability and transferability together. In addition, the proposed method can also be implemented in the physical world. 2.The experiments show that the proposed method is effective.

Weaknesses

1.The novelty of the paper is somewhat limited. The gradient-based adversarial attack method has already been proposed [1]. However, the authors have not cited the paper. 2.In the area of adversarial attacks, it is essential to improve the transferability of the adversarial attack. Although the proposed method can enhance the transferability, the authors has never explained the reason of that. [1] DiffProtect: Generate Adversarial Examples with Diffusion Models for Facial Privacy Protection

Questions

Does the diffusion model necessary in the pipeline? If we use other adversarial purification method in the pipeline, will the results will be better?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Lack the experiment result on the adversarial robust models.

Reviewer PJE26/10 · confidence 3/52023-07-07

Summary

This paper proposes a novel Diffusion-Based Projected Gradient Descent (Diff-PGD) framework which optimizes perturbations by using an off-the-shelf diffusion model. The proposed method works well on both digital world and physical world with high stealthiness.

Strengths

1. The proposed method is novel and performs well on both digital world and physical world. 2. The method analysis is clear and visualization is helpful. 3. The experiments are solid and comprehensive.

Weaknesses

1. Lack of the efficiency report of the proposed method 2. Lack of the performance of proposed method under different L-p constraint. 3. Lack of the performance on transformer based vision models.

Questions

1. I am curious the computation cost of the proposed method. 2. How is the performance of the proposed method on transformer based vision models and under different L-p constraint?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

1. Test the computation time or memory cost of the proposed method. 2. Test the proposed method under different L-p constraint. 3. Test the proposed method on vision transformers.

Reviewer B84v2023-08-11

Thank you for the reply. The rebuttal partially answered my questions.

Thank the authors for their reply and additional results. I have carefully read the author's responses and the comments from other reviewers. The rebuttal answered some of my questions. 1. About my first comment on the marginal improvement over PGD attack: I am aware that both methods achieve 100% attack success rates when n>5, but I was referring to the low-iteration regime, where Diff-PGD performs less effectively. Requiring more iteration makes the method even less efficient. Nonetheless, I agree with the author’s argument that an attack can be solely used for evaluating defense strategies and doesn’t have to be efficient. 2. The second aspect of my concern is the lack of scales in evaluations. I thank the author for conducting additional experiments on 2.5K images. With the new results on 2.5K images, I now tend to believe that the SR would still be 100% on the full set. 3. The quantitative comparison with AdvCAM partially answered my question, but I find the scale of this comparison to be very small (only on 15 pairs of images, if I understand correctly), so the conclusion doesn't seem to be well founded. In the AdvCAM paper, they seem to report their quantitative results on 2000 ImageNet images. I would expect a meaningful quantitative comparison to be conducted on a similar scale. And a follow-up question about the qualitative results in Fig.4: I wonder why your AdvCAM results (the lamp in the bottom row) look much worse than those in Fig.5 of the original paper [1]. 4. About my last comment on the lack of comparison with GAN-based methods. It is important to expand the discussion on related work and why they are not considered in this paper. Although the authors mentioned some drawbacks of existing methods in the response, I don’t think they are solid reasons for not conducting the comparisons (especially for digital attacks). On the contrary, it would further support the proposed method by showing its advantage over the GAN-based methods. Therefore, my concern regarding this aspect remains. Thanks again for the efforts in rebuttal. Please see my comments and follow-up questions in the third point. I'd be willing to raise my score if the authors can help address my concerns and questions. [1]. Duan et al. “Adversarial Camouflage: Hiding Physical-World Attacks with Natural Styles” CVPR 2020.

Authorsrebuttal2023-08-13

Thanks for your reply. We hope we can further address your concern here.

Thanks a lot for your valuable reply, we are glad to answer the questions you further mentioned here, we hope it will further address your concern: > Q1: Yes, Diff-PGD need more steps within the same budget to success, since most PGD attacks perform with famous settings like PGD-10 and PGD-20, we think that Diff-PGD is still an effective attack. > Q2: Thanks for your reply. We are happy to see that we address your concerns on this. > Q3: quantitative comparison with AdvCAM First, in the AdvCAM paper, they report the results (Section 4.2.3 of [1]) on 2000 samples ImageNet dataset as an ablation study of hyper-parameter $\lambda$. They first sample 2000 samples in ImageNet, define the size of the region to attack, and then choose images from the same category as a style reference. The drawback is that the attacked regions are defined simply as squares or circles. In the experiment part (Section 4.3) of [1], they did the style-based attack on around 100 selected samples. To further clarify it, we further construct a better style-based attack dataset based on ImageNet containing 200 cases with 4 categories (sport_car, street_sign, daisy, and guitar), each containing 50 images. We generate the mask using Grounded Segment Anything (Grounded-SAM) [9] using the category text of ImageNet as a prompt, which can effectively segment out the object to be attacked. Following AdvCAM, we also use randomly sampled images of the same category as a style reference. The quantitative results are as follows: |Method|sport_car|daisy|street_sign|guitar| |---| -- | -- | -- | -- | |Diff-PGD|98|100|100|98| |AdvCAM|96|98|94|96| Moreover, we find that the samples generated by Diff-PGD consistently exhibit a high level of quality. To provide a quantitative comparison, we plan to put figures in the final versions of our appendix. Finally, we hope it can address the concern about the scale of quantitative results. > Q3: compared with raw AdvCAM For the results of the lamp shown in Figure 5 of [1], the $\lambda$ was set to 2000 as it was mentioned. However, $\lambda=2000$ cannot always guarantee a high success rate (demonstrated in Figure 6 of [1]). In Section 4.2.1 of [1], it is suggested that we should sample from [1000, 10000] for better $\lambda$. With larger $\lambda$, the generated sample will have larger artifacts (e.g. noisy edges and a stippled background). During our experimental investigations, we adopted a value of $\lambda=4000$ (also set as default in their official GitHub repo) to reliably achieve a high success rate. This decision contributes to the discrepancy between our AdvCAM results and those featured in Figure 5 of the original publication [1]. Furthermore, these observations underscore the necessity for careful hyper-parameter selection in the case of AdvCAM, which may vary across cases. This inherent variability reduces controllability and adaptability compared to Diff-PGD in style-based attacks. > Q4: Thanks for your reply. In the context of digital attacks, GAN-based techniques diverge significantly from conventional approaches, which is why they are typically excluded from the baseline [10, 11]. The rationale behind this exclusion is outlined below: Firstly, a majority of GAN-based methods center around the concept of adversarial training, involving a generator and a discriminator [2, 3, 4]. Importantly, these methods are not **training-free**; they necessitate training over the dataset. Moreover, GAN-based approaches do not notably excel in terms of stealthiness, as evidenced by findings in [2, 3, 4]. Also, there are other studies, such as [5], that employ GANs for producing **unrestricted attacks**; however, these fall beyond the scope of our task. For physical-world attacks where GAN-based achieved great success. especially imethods such as [6, 7] optimize from random noise to sample from GAN as adversarial pattern, while our pipeline focuses more on cases with image reference, so we only consider methods with a reference image as input. Still, we thank the reviewer for this question, we will better clarify why GAN-based methods are not included as baselines in our paper. [1] Adversarial Camouflage: Hiding Physical-World Attacks with Natural Styles [2] AI-GAN: Attack-Inspired Generation of Adversarial Examples [3] Generating Adversarial Examples with Adversarial Networks [4] A two-stage generative adversarial networks with semantic content constraints for adversarial example generation [5] Constructing Unrestricted Adversarial Examples with Generative Models [6] Naturalistic physical adversarial patch for object detectors. [7] Advart: Adversarial art for camouflaged object detection attacks [8] Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models [9] Segment Anything [10] Diffusion Models for Imperceptible and Transferable Adversarial Attack [11] Content-based Unrestricted Adversarial Attack

Reviewer B84v2023-08-14

Thanks for the clarification. I find the additional larger-scale experiments to better support the proposed method, and I recommend authors report the updated results in the next version of the paper. I'm raising my score accordingly.

Authorsrebuttal2023-08-14

Thanks again for your valuable feedback

Thank again you for the valuable feedback. We will try to add the related discussions and analysis in our revision, and please let us know if you have further suggestions!

Reviewer PJE22023-08-16

The anthor answers most of my questions well and I will keep my rating.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC