Diffusion-based purification defenses leverage diffusion models to remove crafted perturbations of adversarial examples and achieve state-of-the-art robustness. Recent studies show that even advanced attacks cannot break such defenses effectively, since the purification process induces an extremely deep computational graph which poses the potential problem of gradient obfuscation, high memory cost, and unbounded randomness. In this paper, we propose a unified framework DiffAttack to perform effective and efficient attacks against diffusion-based purification defenses, including both DDPM and score-based approaches. In particular, we propose a deviated-reconstruction loss at intermediate diffusion steps to induce inaccurate density gradient estimation to tackle the problem of vanishing/exploding gradients. We also provide a segment-wise forwarding-backwarding algorithm, which leads to memory-efficient gradient backpropagation. We validate the attack effectiveness of DiffAttack compared with existing adaptive attacks on CIFAR-10 and ImageNet. We show that DiffAttack decreases the robust accuracy of models compared with SOTA attacks by over 20% on CIFAR-10 under $\ell_\infty$ attack $(ε=8/255)$, and over 10% on ImageNet under $\ell_\infty$ attack $(ε=4/255)$. We conduct a series of ablations studies, and we find 1) DiffAttack with the deviated-reconstruction loss added over uniformly sampled time steps is more effective than that added over only initial/final steps, and 2) diffusion-based purification with a moderate diffusion length is more robust under DiffAttack.
Paper
Similar papers
Peer review
Summary
This paper aims to generate adv-samples that can effectively attack diffusion-based purification. Two key strategies are proposed: (a) deviated reconstruction loss which is used as a surrogate loss to relieve gradient obfuscation existing in the long computational graph; (b) segment-wise forward-backward strategy is applied to relieve the high memory cost to calculate accurate $L_{cls}$. The experiments show that the method proposed is effective.
Strengths
Overall, the paper is well-written and easy to read, the clarification is clear and - The problem is important since diffusion-based purification is proven to be effective and model-agnostic. - The proposed deviated reconstruction is proven to be effective. - The segment-wise forward-backward is effective and well-design - The experiments results are extensive.
Weaknesses
I mainly have some questions here: - Will the segment-wise forward-backward affect the speed compared with the original forward-backward? - Intuitively, the adv-samples generated by DiffAttack are (1) hard to be purified by one specific DM, (2) hard to denoise at each stage in a given DM. When the budget $\epsilon$ is larger, e.g. 8, 16 for imagenet, what will the adv-samples be like? - The code is not provided in the supp materials, it will be better to show some demo code - Do well need to carefully design the $\lambda$ weights for the two losses? - (Some related paper) Can DiffAttack also transfer better to other networks? This topic is quite related to some recent papers, which prove that only attacking the diffusion model can be transferred well. [1, 2] I am willing to raise my score if most of the questions are solved. [1] Chen, Z., Li, B., Wu, S., Jiang, K., Ding, S., & Zhang, W. (2023). Content-based Unrestricted Adversarial Attack. arXiv preprint arXiv:2305.10665. [2] Xue, H., Araujo, A., Hu, B. and Chen, Y., (2023). Diffusion-Based Adversarial Sample Generation for Improved Stealthiness and Controllability. arXiv preprint arXiv:2305.16494.
Questions
refer to `Weaknesses`
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
NULL
Summary
This paper proposes a new adaptive attack to unify the robustness evaluation of diffusion-based purification defenses, including both score-based and DDPM-based diffusion models. The new attack mainly includes two novel techniques to overcome the challenges of evaluating diffusion-based purification defenses. The first technique is a regularizer that encourages a large distance between the forward images and their counterparts in the reverse process, which is proposed to tackle the vanishing or exploding gradients problem. The second technique is a forward-backward algorithm that allows for gradient back-propagation segment by segment, which is proposed to reduce the high memory cost. Evaluations on CIFAR-10 and ImageNet show that the proposed adaptive attack can lower the robust accuracy by a large margin while maintaining low memory costs.
Strengths
### Originality * **Novel attack techniques.** The overall attack is novel. The proposed regularizer is not surprising but requires some insight to apply properly. The segment-wise forward-backward algorithm is also novel to my knowledge. ### Quality * **Comprehensive evaluations.** The evaluation includes two models on CIFAR-10 and three models on ImageNet, with two purification defenses and several baseline adaptive attacks. * **Good ablation studies.** Other studies of the diffusion lengths, memory cost, and the proposed loss term are relatively sufficient. * **The theorem unifies attacks against score-based and DDPM-based purification defenses.** The presented two theorems show that increasing the new loss term can increase the data estimation loss of both DDPM-based and score-based purification defenses, although these theorems are not sufficiently discussed or motivated. ### Clarity N/A ### Significance * **Timely topic.** Diffusion-based purification is a fast-developing direction in adversarial example defenses. Given the well-known challenges and the lacked systemized evaluation of these defenses, this paper tackles an important problem in adversarial example defenses. * **Strong results.** It is good to see that the proposed attack can lower the robust accuracy by around 10% to 20%. The memory cost is also very low compared with previous approaches.
Weaknesses
### Originality **1. [Moderate] Missing related work for memory-efficient gradient back-propagation.** One major contribution of this paper is the memory-efficient gradient back-propagation algorithm. However, this paper did not discuss this literature and potential previous approaches for improving memory efficiency, either for the general optimization problem or for the diffusion model specifically. It is also claimed that this algorithm *"can be used for any discrete Markov process."* This is a strong claim and seems to contribute to a broader area. Yet, without a solid discussion of related work, it is unclear how to position this algorithm in the literature. For example, is there any existing memory-efficient algorithm that is also able to tackle the memory overheads when evaluating diffusion-based purification defense? If so, a further ablation study may be necessary to show why the algorithm in this paper is still needed. If not, it is recommended to highlight this contribution more. **2. [Minor] Attack framework vs. attack techniques.** The proposed methods are more like novel attack techniques rather than an attack framework. As far as I can tell from Algorithm 2, the attack generally follows the existing PGD or APGD attack framework (gradient update + projection). It is suggested to tune down this claim if the authors could not differentiate the proposed attack's framework from existing ones. ### Quality **3. [Major] Did not address the motivated challenge of unbounded randomness.** The authors called out the *"unbounded randomness"* problem in many places throughout the paper (e.g., lines 6, 32, 43, 133, 226, 391). However, the paper did not explain how this problem was resolved by the two proposed techniques. The missing explanation is confirmed by the summarized technical contributions in L69-83, where the two proposed techniques both tackle one of the remaining two challenges. There are two places discussing the randomness problem but likely for some other contexts: the expectation in Equation (8) and the optimization decision in L234-240. However, neither of these discussions could explain the "unbounded randomness" challenge in the original evaluation. Please clarify if the proposed attack could tackle the randomness problem. If not, it is suggested to discuss why the proposed attack does not need to tackle the randomness problem to obtain strong results. If that is indeed the case, I am curious if the results in this paper would connect to the recent studies showing that randomness might be unhelpful in defenses [A, B]. Otherwise, is it possible to lower the robust accuracy even more if the randomness problem is explicitly tackled? Combined together they could strengthen this paper by showing some insights into the most challenging problem in evaluating purification defenses (gradient, memory, or randomn[[2023-06-28]]ess). [A] On the Limitations of Stochastic Pre-processing Defenses. NeurIPS 2022. [B] Randomness in ML Defenses Helps Persistent Attackers and Hinders Evaluators. arXiv 2023. **4. [Major] It is unclear how the gradient issue was resolved.** The authors claimed that the proposed deviated-reconstruction loss in Equation (8) could resolve the gradient problems, including both gradient obfuscation (L48, L72, etc.) and gradient vanishing/exploding (L10, L141, etc.). However, it is unclear how these problems are resolved by the proposed loss term. This issue has three folds. 1. **[Minor] It is unclear what exact problem is being resolved** when the authors mixed the concepts of "gradient obfuscation" and "vanishing/exploding gradients." Specifically, "gradient obfuscation" is a general concept that also includes non-differentiable defense components, where the gradient is undefined and hence zero, yet "vanishing/exploding gradients" specifically refers to the defined but overly small or large gradients in differentiable defense components. While these are minor conceptual confusion, it is suggested to be consistent in the wording and clarify if it is the "non-differentiable" or "differentiable but vanishing/exploding" problem that the technique aims to solve, as their solutions typically require different strategies. 2. **[Moderate] It is unclear if the gradient issue indeed exists in previous evaluations.** While it is true that deep networks could have the problem of vanishing or exploding gradients, they have not been explicitly observed by previous evaluations of purification-based defenses, such as DiffPure [29]. Given this, it is hard to justify if the proposed techniques could obtain stronger results because they have indeed resolved the gradient issues or some other undiscovered reasons. 3. **[Major] It is unclear why adding a loss term B would resolve the gradient problem of the existing loss term A.** Specifically, if there were a gradient problem in the original robustness evaluation (with only $\mathcal{L}\_{cls}$), it means that the loss term $\mathcal{L}\_{cls}$ has already exhibited some form of gradient obfuscation (be it non-differentiable or vanishing gradients). In this case, why would adding a new loss term $\mathcal{L}\_{dev}$ would be able to resolve the gradient problem in $\mathcal{L}\_{cls}$? For example, if the total loss shows non-vanishing gradients, it is very likely that the gradient of $\mathcal{L}\_{cls}$ is still vanishing, and the observed gradient solely comes from the added $\mathcal{L}\_{dev}$. While it is explained how $\mathcal{L}\_{dev}$ can relieve the gradient problem (L153), it only explains how $\mathcal{L}\_{dev}$ relieve the gradient problem of itself but not $\mathcal{L}\_{cls}$. **5. [Major] Unclear effectiveness of the proposed regularizer.** Given that it is unclear how the gradient issue was resolved, I feel the effectiveness of $\mathcal{L}\_{dev}$ should require more exploration from two perspectives. 1. If it indeed solves the gradient problem of $\mathcal{L}\_{cls}$, the authors should provide evidence showing that the problem (1) exists before adding $\mathcal{L}\_{dev}$, and (2) disappears after adding $\mathcal{L}\_{dev}$. 2. If it cannot solve the gradient problem of $\mathcal{L}\_{cls}$ but mostly optimizes the gradient problem of itself, then it seems to me that making forward-backward samples deviated is more effective than increasing the classification loss, which is an interesting insight. In this case, I would recommend the authors explore the rationale behind this observation, and highlight the efforts of resolving gradient problems in $\mathcal{L}\_{dev}$ rather than $\mathcal{L}\_{cls}$. Additionally, an ablation study of the loss weight $\lambda$ should be included. **6. [Moderate] Unclear discussion about randomized purification.** The discussion at L234 is unclear (even with L674-676). * First of all, if a single prediction has a high variance, the ultimate prediction should be an ensemble of multiple predictions, e.g., through majority vote. After that, it seems reasonable for attacks not to evaluate misclassified samples. From my understanding, this choice is not to reduce the computational cost, but a fact that you do not need to attack it -- you already find an adversarial example with distance zero. * I did not see the logical connection between L234-L237 and L237-L240, mostly because it is unclear if the first part refers to the fixed clean image or the sampled images during the attack. * Also, the effectiveness of this choice was not evaluated. Is this choice a must-have for the final results? **7. [Minor] Visualization of purified noise.** Since this paper was able to circumvent purification defenses, it is suggested to also visualize the adversarial noise before and after the purification defense. Right now only the pre-purification adversarial examples are provided. ### Clarity **8. [Major] The presentation is confusing and contrived.** * Be consistent when describing the gradient problem (detailed in Quality-4.1). The current presentation uses "gradient obfuscation" and "vanishing/exploding gradients" interchangeably, but the previous term is a general concept that also includes non-differentiable problems. In Section 3.2, jumping between these two terms and the proposed technique is very distracting. It is suggested to use only one of them and be clear about what specific problem you are trying to solve. * The intuition behind Equation (8) is not clear until paragraph L166. It is suggested to explain the intuition (deviation, not the gradient problem) first. * The paragraph L157 comes out of the blue and is very dense, it might fit more at the end of Section 3.2. * Theorem 1 also comes out of the blue. The use of this theorem is unclear until L189. It is suggested to explain the remark (L187) or at least its high-level idea before introducing the theorem. * The use of Theorem 3 was presented without any discussion. * It is a bit hard to determine what exact defense was evaluated, it would be good to specify the defense name rather than citations. **9. [Minor] Typos and notations.** * Undefined terms * L151, $d()$ * L172, "data density estimation" * L177, VP-SDSE * L206, $f\_d$ and $f\_r$ * Typos * L142, Path -> Pass * L152, sampled -> reconstructed. ### Significance N/A
Questions
I am willing to raise my score if the following questions are adequately answered or clarified, ordered by importance. 1. Quality 4, 5 2. Quality 3 3. Quality 6 (merge to Quality 3 if they are relevant) 4. Originality 1 5. Quality 7 6. Fix Clarity 8
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
2 fair
Presentation
1 poor
Contribution
2 fair
Limitations
N/A
Summary
This paper proposes a framework, DiffAttack, to attack against diffusion-based adversarial purification defenses. To address the issue of gradient vanishing/exploding, the paper introduces the concept of deviated-reconstruction loss and theoretically analyzes its relationship with density gradient estimation. To overcome the challenge of high memory cost, the paper presents a segment-wise forwarding-backwarding algorithm. Empirical experiments demonstrate the effectiveness of the proposed method and provide observations that contribute to a better understanding of the properties of the diffusion process.
Strengths
1. The content is rich. 2. The theoretical analysis is solid. 3. The experimental results show improvements compared to the competing algorithm.
Weaknesses
1. Section 3.2 discusses that maximizing the deviated-reconstruction loss in Equation (8) can lead to inaccurate data density estimation, which results in a sampling distribution that is inconsistent with the clean training distribution. However, it does not provide a detailed explanation of how this helps alleviate gradient vanishing/exploding. 2. Additional experiments should be conducted to investigate the transferability of DiffAttack.
Questions
1. Line 151 does not provide a specific equation for α(·). 2. The paragraph from lines 157-165 should be placed before the paragraph from lines 147-156.
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
4 excellent
Presentation
3 good
Contribution
3 good
Limitations
NA
Summary
The paper introduces an adversarial attack focused on diffusion-based purification techniques. In diffusion-based purification, a (malicious) input is first noised and then denoised by a diffusion model before being fed through the classifier. The paper aims at solving the issues that arise when using backpropagation-based PGD attacks due to the size of the denoising diffusion graph. The two main contributions are: - A per-timestep deviated reconstruction loss that maximizes the distance between the noisy version of the original image at a certain timestep and the denoising reconstruction. - Segment-wise forwarding-backwarding which is essentially gradient checkpointing for diffusion graphs In the evaluation. they test their method against a variety of SOTA baselines on CIFAR10 and ImageNet and are able to beat previously best achieved robust accuracies by a margin.
Strengths
- The evaluation protocol seems reasonable to me and the results are good across the board compared to the given baselines. - Overall the presentation is clear - Both gradient checkpointing and the deviated reconstruction loss are quite simple ideas that seem to work well at solving the problem at hand.
Weaknesses
- I think the formulation of the segment-wise forwarding-backwarding algorithm makes it seem more novel than it truly is. Gradient checkpointing has been used to compute gradients in deep computational graphs by trading memory for compute in a lot of papers. I agree that diffusion graphs are well suited for this approach as the diffusion latents that are stored are very compact in size compared to the activations inside the score-net, but I still believe that it would be clearer to the reader to just refer to it as gradient-checkpointing. - I personally find some arrows in Figure 1 to be a bit confusing. I do not really understand the red arrows DiffAttack in the top part of the graph at t=0 and it's not super clear how they interact with the grey arrows for segment-wise forwarding-backwarding. I understand how the method works from the description in the text and I think it's a good idea to have a figure illustrating gradient flow in the paper but I believe clarity could be improved here. - In my opinion, Section 3 should contain a brief description of diffusion-based purification defenses in mathematical notation as the paper is explicitly aimed at attacking them. - I think it should be clearly stated at (11) that the goal is to maximize the objective similar to (6)/(7). Maybe it would even improve clarity to add a max close to (8) as minimization is more common in deep learning literature - I like Section 4.7 but I think it could be improved by showing results without the added deviated-reconstruction loss at all. - I believe the theoretical connection between Theorem 1 and the rest of the paper is rather weak. The Theorem only connects total variation to a mismatch between the ground-truth score function and the network estimate of the score. I think this would be a lot more useful if the paper would actually maximize the total-variation distance or if the Theorem could be stated in terms of the quantity being maximized.
Questions
- Especially in classifier guidance, it is common to denoise the current latent x_t to get a coarse estimate of x_0 and then calculate the gradient through this single-step approximation (For example "UPainting: Unified Text-to-Image Diffusion Generation with Cross-modal Guidance" Li et Al 2022 ). You mention several strategies in 3.2 to prevent gradient obfuscation by introducing additional losses at intermediate steps. Would it be possible to apply such an approach to your problem and do you expect it to work better/worse than your deviated-reconstruction loss?
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
3 good
Contribution
2 fair
Limitations
- I think the paper is very applicable to diffusion-based purifications and the authors do not mention strong limitations either.
Thanks for the response and additional experiments. I have raised my score to 5 with the following two concerns. **Q4 & Q5: Regarding how the added loss term resolves the gradient issue.** I generally agree with the response regarding how the added loss term suffers less from the gradient problem. This conclusion, however, points out the non-usefulness of the original classification loss, which is an important observation (to me) but not addressed in the paper --- It seems that simply making the reconstructed image more deviated would suffice to reduce the defense's robustness, as least from the current presentation that there is a gradient problem with the classification loss (which is not explicitly demonstrated tho) and this paper does not resolve that directly. I have no problem with the proposed loss, but I feel it is important to make the above point clear in the paper. For example, would you be able to obtain a similar success rate if the classification loss was removed completely? If this is demonstrated (either true or false), the overall clarity (and potential contribution if demonstrated true) of this paper would be enhanced a lot. Beyond this, I am fine as long as the revised paper clarifies that the proposed loss does not resolve the gradient problem in the classification loss directly, but somehow circumvents that problem through a proxy. As a side suggestion, a more comprehensive evaluation could be showing the curves for the two loss terms, so readers would understand what is going on with the two losses. **Q3 & Q6: Regarding the unbounded randomness.** I agree with the two perspectives clarified in the rebuttal, which I already noted in my initial Q3. I think a better way to put my concern is that the current argument (for its novelty or importance) is a bit weak based on the following two points. First, why the randomness is "unbounded?" Previous defenses like DiffPure were also evaluated with EOT [A], which has the same spirit as Equation (8) in this paper, so I cannot see evidence that their randomness is not bounded yet this paper is bounded. This leads to the second point -- how Equation (8) is different from previous evaluations (in terms of resolving randomness)? I feel the root improvement is more relevant to the gradient problem of the original classification loss (rather than the design choices for resolving randomness). For example, it might be the case that EOT + the proposed loss is more useful than EOT + the original classification loss. Given the above two points, I feel the significance of resolving randomness can be tuned down in the paper (if I understand correctly). The most interesting part is the technical details of (1) how you apply EOT to the new proposed loss, and (2) what space you apply EOT. I think separating these discussions from the two main techniques could make the contributions more clear. Right now the randomness is presented as a novel problem (which is not the case as discussed above), yet the corresponding designs are not that significant (to me, given the previous application of EOT). I believe tuning down this part would not hurt the overall contribution a lot, since the other two parts are strong enough. [A] Synthesizing Robust Adversarial Examples. ICML 2018.
Follow-up Discussions with Reviewer yE5n [1/2]
> Q1: Regarding how the added loss term resolves the gradient issue. Thank you for the valuable feedback and suggestions! We add additional evaluations following the suggestion to explore whether the deviated-reconstruction loss suffices to reduce the defense’s robustness. We perform $\ell_\infty$ attacks against score-based purification with pretrained classifier WideResNet-28-10 on CIFAR-10. The results in Table 1 demonstrate that **only the deviated-reconstruction loss is not sufficient to attack the diffusion purification defenses**. We will clarify that although the classification loss suffers more from the gradient issue, it provides the necessary guidance to optimize the sample across the decision boundary. In the revised version, we will clarify that *the deviated-reconstruction loss does not resolve the gradient problem of the classification loss directly but circumvents the problem by leading to a deviated reconstruction of the image, which is easier to be misclassified and thus guides the optimization of searching adversarial examples effectively*. We will add the detailed explanations and discussion of the functionality of the deviated-reconstruction loss in responses to Q4 & Q5 in our rebuttal following the suggestion to make it clear. Thanks for the suggestions! Table 1: Attack performance (Robust accuracy (%)) of applying only deviated reconstruction loss (+$\ell_{dev}$), only classification loss (+$\ell_{cls}$), and the combinations of them (+$\ell_{cls}$+$\ell_{dev}$) against score-based purification with pretrained classifier WideResNet-28-10 on CIFAR-10 with different perturbation budgets $\epsilon$. | | +$\ell_{dev}$ | +$\ell_{cls}$ | +$\ell_{cls}$+$\ell_{dev}$ | | -------------------------------------------------- | ------------------ | ---------------- | ---------------- | |$\epsilon=8/255$ | 78.13 | 70.64 | 46.88 | | $\epsilon=4/255$ | 85.16 | 82.81 | 71.88 | We also evaluate the curves for the two loss terms (i.e., deviated-reconstruction loss $\ell_{dev}$ and cross-entropy loss $\ell_{cls}$) with score-based purification with WideResNet-28-10 on CIFAR-10 under $\ell_\infty$ DiffAttack with $\epsilon=8/255$. Comparing the loss curve of $\ell_{cls}$ without applying $\ell_{dev}$ (in Table 2) and the loss curve of $\ell_{cls}$ with applying $\ell_{dev}$ (in Table 3), we can see that the introduction of $\ell_{dev}$ guides the optimization of $\ell_{cls}$ effectively. We will make the results more clear by presenting them in figures in the revised version. Table 2: Loss curves of the cross-entropy loss $\ell_{cls}$ by applying only the classification loss $\ell_{cls}$ to attacking score-based purification with WideResNet-28-10 on CIFAR-10 under $\ell_\infty$ DiffAttack with $\epsilon=8/255$. | Epoch | 0 | 5 | 10 | 15 | 20 | 25 | 30 | 100 | | - | - | - | - | - | - | - | - | - | | $\ell_{cls}$ | 0.3856 | 0.6384 | 0.7283 | 0.7728 | 0.8028 | 0.8092 | 0.8091 | 0.8293 | Table 3: Loss curves of deviated-reconstruction loss $\ell_{dev}$ and cross-entropy loss $\ell_{cls}$ by applying $\ell_{dev}$ and $\ell_{cls}$ to attacking score-based purification with WideResNet-28-10 on CIFAR-10 under $\ell_\infty$ DiffAttack with $\epsilon=8/255$. | Epoch | 0 | 5 | 10 | 15 | 20 | 25 | 30 | 100 | | - | - | - | - | - | - | - | - | - | | $\ell_{cls}$ | 0.3856 | 0.8237 | 1.0289 | 1.1782 | 1.2843 | 1.3582 | 1.3726 | 1.3832 | | $\ell_{dev}$ | 0.2685 | 0.6823 | 0.8394 | 0.9523 | 1.0383 | 1.1353 | 1.1534 | 1.1634 |
Follow-up Discussions with Reviewer yE5n [2/2]
> Q2: Regarding the unbounded randomness. Thank you for the valuable suggestions! We will follow the reviewer’s suggestions and tune down the significance of solving randomness and provide more details of the optimization process and analysis. We will illustrate how DiffAttack benefits resolving the randomness problem in detail as follows. First, we will point out that attacking against diffusion purification defenses suffers from the problem of large randomness because the diffusion and sampling process introduces large randomness which makes the calculated gradients unstable and noisy. Specifically, the randomness comes from the addition of Gaussian noises at each step in the diffusion process and sampling process. The randomness is large since there are typically multiple steps of Gaussian noise addition (e.g., 100 or more for score-based diffusion purification in DiffPure). We will rephrase “the problem of unbounded randomness’’ as “the problem of large randomness’’ in the abstract to avoid confusion. Second, we will clarify the strategy of approximating the expectation in Equation (8) by sampling multiple times and taking average benefits resolving the randomness problem similarly to EOT [A]. They both reduce the approximation error of the mean of random gradients by increasing the sample sizes. However, the benefits of the deviated-reconstruction loss in terms of the randomness challenge lie in the following perspective. When we expand the purification process as the computation graph, there is more randomness introduced at deep layers compared with shallow layers because more Gaussian noises are added at deeper layers. The deviated-reconstruction loss is applied uniformly across different layers of the computation graph and there is less randomness at shallow layers, while the classification loss is only added at the deepest layer of the graph and there is more randomness introduced at deep layers. Therefore, the deviated-reconstruction loss suffers less from the randomness problem, and the deviated-reconstruction loss with multiple samplings (EOT) outperforms the classification loss with multiple samplings (EOT). We will also include more technical details of applying EOT to the deviated-reconstruction loss following the suggestion. In particular, for a given batch of data, we can perform the diffusion process and sampling process and compute the gradient of the deviated-reconstruction loss with respect to the input data. We repeat the process multiple times and compute the average of the gradient across multiple times to estimate the mean of the gradient of deviated-reconstruction loss. We will follow the reviewer’s suggestions to add these discussions in the revision and update our claims to make the contributions more clear. *[A] Synthesizing Robust Adversarial Examples. ICML 2018.*
Thanks for the response. The new results look promising and insightful. I have raised my score to 6 accordingly.
Thank you for the valuable feedback
Thank you for the valuable feedback to help improve our work. We will definitely add the related discussions and analysis in our revision, and please let us know if you have further questions or suggestions!
Thanks for the rebuttal
Thanks for providing the rebuttal. I will keep my original rating and recommend acceptance.
Thanks for the reply
I have read the rebuttal by the authors, and I have also read the comments of other reviewers. I would like to remain my score and suggest a Borderline acceptance.
I would like to start by thanking the authors for the rebuttal. - Q1: I do agree that previous works focus on "inner network checkpointing" whereas this method checkpoints between the solver steps. I believe it's a valid contribution. I just think it would be easier to understand the idea for people that know checkpointing by shortly referencing it. - Q2 - Q4: Thank you for these updates and improving clarity! - Q5: Thank you for the experiments. These results show that the additional regularization term helps a lot with the problem at hand. - Q6: I still believe that there is a certain gap between the theoretical results and the practical formulation but I understand that it is not possible to actually calculate the TV distance. Theoretical insights that directly analyze the connection with the quantity that is optimized would be ideal though. Overall, the paper shows some strong results on a relevant problem. The method itself is not super novel but effective and I believe the paper is worth acceptance if some of the clarity issues from Q2-Q4 are resolved.
Follow-up Discussion with Reviewer 3df8
Thank you for the valuable feedback and suggestions! We will reference the gradient checkpointing technique when introducing the segmentwise forwarding-backwarding algorithm in Section 3.3 for better understanding and also incorporate the related work of gradient checkpointing in Section 5 for completeness. Thank you for the suggestions! We will definitely include all the modifications in Figure 1, equation 7, and equation 11 for better clarification in the revised version. We will also replace Table 5 in the manuscript with Table 10 in the one-page rebuttal PDF for a better presentation of the results. Thanks for the valuable comments about Theorem 1. We plan to further reduce the gap between the analysis and practical formulation following the suggestion as follows. In Theorem 1, $q \_t$ is a Gaussian distribution with the mean $\tilde{x} \_t$ and $q’ \_t$ is a Gaussian distribution with the mean $\tilde{x}’ \_t$. We analyze the objective $D_{TV}(q \_t,q’ \_t)$ in Theorem 1 and show that maximizing it can induce inaccurate data density estimation and thus inaccurate sample reconstruction. In the deviated-reconstruction loss, we maximize the vector distance between $\tilde{x} \_t$ and $\tilde{x}’ \_t$, which are the mean of Gaussian distribution $q \_t$ and $q’ \_t$, respectively. The connection of the two objectives can be built by providing more theoretical insights about why maximizing the distance of mean of Gaussians (i.e., $|\tilde{x} \_t - \tilde{x}’ \_t|$) can implicate maximizing the TV distance between Gaussians (i.e., $D_{TV}(q \_t,q’ \_t)$). Theorem 1.3 in [a] shows that for one dimensional Gaussian $\mathcal{N}(\mu \_1,\sigma \_1^2)$ and $\mathcal{N}(\mu \_2,\sigma \_2^2)$, the TV distance can be lower bounded by a function of the distance of mean values as the following: $$ \dfrac{1}{200} \min \\{1, \max \\{ \dfrac{40|\mu \_1-\mu \_2|}{\sigma \_1}, \dfrac{|\sigma \_1^2-\sigma \_2^2|}{\sigma \_1^2} \\} \\} \le D_{TV}(\mathcal{N}(\mu \_1,\sigma \_1^2),\mathcal{N}(\mu \_2,\sigma \_2^2)), $$ from which we can see that maximizing the distance of the means (i.e., $|\mu \_1-\mu \_2|$) can indicate maximizing the TV distance $D_{TV}(\mathcal{N}(\mu \_1,\sigma \_1^2),\mathcal{N}(\mu \_2,\sigma \_2^2))$. Similar insights are shown for multi-dimensional Gaussians in Theorem 1.2 in [a]. We will add the theoretical intuitions and related discussions in Theorem 1 in Section 3.2 for clarification following the suggestions. We are glad that the reviewer finds our method effective with strong results and worth acceptance. Please let us know if there are other comments and suggestions. Thank you for your valuable suggestions again! *[a] Devroye, Luc, Abbas Mehrabian, and Tommy Reddad. "The total variation distance between high-dimensional Gaussians with the same mean." arXiv preprint arXiv:1810.08693 (2018).*
Decision
Accept (poster)