Content-based Unrestricted Adversarial Attack

Unrestricted adversarial attacks typically manipulate the semantic content of an image (e.g., color or texture) to create adversarial examples that are both effective and photorealistic, demonstrating their ability to deceive human perception and deep neural networks with stealth and success. However, current works usually sacrifice unrestricted degrees and subjectively select some image content to guarantee the photorealism of unrestricted adversarial examples, which limits its attack performance. To ensure the photorealism of adversarial examples and boost attack performance, we propose a novel unrestricted attack framework called Content-based Unrestricted Adversarial Attack. By leveraging a low-dimensional manifold that represents natural images, we map the images onto the manifold and optimize them along its adversarial direction. Therefore, within this framework, we implement Adversarial Content Attack based on Stable Diffusion and can generate high transferable unrestricted adversarial examples with various adversarial contents. Extensive experimentation and visualization demonstrate the efficacy of ACA, particularly in surpassing state-of-the-art attacks by an average of 13.3-50.4% and 16.8-48.0% in normally trained models and defense methods, respectively.

Paper

Similar papers

Peer review

Reviewer QZtm6/10 · confidence 5/52023-07-04

Summary

This paper proposes a novel unrestricted attack framework called Content-based Unrestricted Adversarial Attack (ACA). The author argues that current unrestricted attacks have limitations in terms of maintaining human visual imperceptibility, generating natural adversarial examples, and achieving high attack performance. This paper is well-written and easy to follow.

Strengths

1) The paper introduces a novel attack framework that addresses the limitations of current unrestricted attacks. The use of a low-dimensional manifold and optimization along its adversarial direction allows for the generation of diverse and natural adversarial examples. 2) The paper provides a clear motivation and problem definition, outlining the challenges and goals for unrestricted adversarial attacks. 3) The paper includes extensive experimentation and visualization to validate the effectiveness of ACA. The results show significant improvements over state-of-the-art attacks in terms of adversarial transferability.

Weaknesses

The paper does not include sufficient evaluation and ablation studies for the proposed method. Since ACA used skip gradient, I think I should compare Skip connections matter [1] in the experiment. The paper could benefit from a more thorough review of relevant literature. While the authors mention existing unrestricted attacks, there is limited discussion on related work and the novelty of ACA compared to previous approaches. [1] Skip connections matter: On the transferability of adversarial examples generated with resnets, ICLR 2019

Questions

/Na

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

/Na

Reviewer frod5/10 · confidence 4/52023-07-04

Summary

The paper introduces a novel attack framework called Content-based Unrestricted Adversarial Attack, which aims to generate diverse and natural adversarial examples with high transferability. The authors argue that existing methods, such as lp norm-based attacks, have limitations in terms of perceptual similarity, naturalness, and robustness. To address these issues, they propose mapping images onto a low-dimensional manifold represented by a generative model trained on natural images. This manifold ensures both photorealism and content diversity. By optimizing the adversarial objective on this latent space, they generate unrestricted adversarial examples. The proposed method, called Adversarial Content Attack (ACA), utilizes Image Latent Mapping (ILM) and Adversarial Latent Optimization (ALO) techniques to optimize the latent in a diffusion model. The effectiveness of ACA is validated through experiments and visualization, demonstrating significant improvements of 13.3~50.4% in terms of adversarial transferability compared to state-of-the-art attacks. Overall, the main contributions of the paper are the introduction of the Content-based Unrestricted Adversarial Attack framework, the development of the Adversarial Content Attack method, and the experiments demonstrating improvements in generating diverse and transferable adversarial examples.

Strengths

The authors effectively communicated the motivation, problem statement, and methodology of the proposed framework. By addressing the limitations (imperceptibility/photorealism/effectiveness) of existing methods, they proposed a novel attack framework that leverages a low-dimensional manifold represented by a generative model. By combining image mapping onto a latent space, optimizing adversarial objectives, and utilizing a diffusion model, the authors introduce a novel approach to generating diverse and natural adversarial examples. This paper might be the first to explore unrestricted adversarial examples through such a framework. This paper offers a thorough explanation of the proposed attack framework, detailing the underlying techniques of ILM and ALO. The authors further support their claims through experimentation and visualization, providing evidence of the effectiveness of their approach and demonstrating improvements in adversarial transferability compared to state-of-the-art attacks. The improvements in adversarial transferability also shed light on the potential impact of this method in uncovering vulnerabilities in security-sensitive applications and advancing our understanding of robustness in DNNs. Overall, this paper's strengths encompass originality in proposing a novel attack framework, quality in terms of methodology and experimental evaluation, clarity in explaining the concepts and techniques, and significance in addressing limitations and raising awareness of unrestricted but realistic adversarial examples.

Weaknesses

While the paper has several strengths, there are some weaknesses that could be addressed to further improve the work: **Comparison with State-of-the-Art Attacks:** While the paper mentions that the proposed method achieves significant improvements in terms of adversarial transferability compared to state-of-the-art attacks, a more comprehensive comparison would strengthen the evaluation. It would be valuable to include a thorough analysis and comparison with a wider range of existing unrestricted attack methods, such as [1]. In particular, Laidlaw et al. proposed an efficient way to generate imperceptible adversarial examples. The reviewer also suggests evaluating the proposed attack on other adversarially trained models, for example, the defense method that could be generalized to unforeseen perturbations [1], or having used synthetic data during adversarial training [2]. **Defense Method:** This paper does not discuss how to defend the proposed attacks but primarily on the efficacy of the proposed method. It would be great if the authors provide potential solutions or mitigation strategies for the threats. **Generalization to Different Datasets:** The authors only evaluate their method on a subset of the ImageNet validation set and do not say how the results generalize to other datasets. It would be beneficial to investigate the generalization of the proposed approach to different datasets. **Unclear Claim:** The authors mentioned the Dunning-Kruger effect to emphasize that current defense methods against lp norm adversarial examples overestimate their abilities. However, this work does not provide further details and arguments to support this. Although similar arguments have also been proposed in [3], in which the authors argue that lp-based robustness evaluation might be biased, the reviewer thinks that using the Dunning-Kruger effect here is not rigorous. The reviewer suggests that the authors rethink this argument, and even consider removing it if it is not an important contribution of the paper. **Typos:** Although this paper is well-written and easy to follow, the reviewer found some typos and grammar errors. For example, in line 161, *follw*. It would be great if the authors have proofread the paper before submitting it. [1] Laidlaw et al. Perceptual adversarial robustness: Defense against unseen threat models. (ICLR 2021) [2] Croce et al. Robustbench: a standardized adversarial robustness benchmark. (NeurIPS 2021) [3] Hsiung et al. CARBEN: Composite Adversarial Robustness Benchmark. (IJCAI 2022)

Questions

Please refer to the weakness part. In brief, the reviewer would like to understand more about the following points: - Could the authors provide more attack and defense baselines as mentioned in the weakness? - Could the authors provide some experiments on other datasets? - Please address the mentioned unclear claim. - Why the authors did not provide the code for review but answered the Reproducibility as "yes"?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

N/A

Reviewer Rhy16/10 · confidence 3/52023-07-06

Summary

In this paper, authors propose an unrestricted untargeted attack based on optimising the latent space of stable diffusion model. The generated adversarial samples are empirically shown to be more transferable than the existing semantic attacks. Moreover, authors validate the effectiveness of adversarial samples when attacking different representative adversarial trained models and show consistent performance book.

Strengths

- The method is a replacement of popular GANs in the semantic attacks with the powerful stable diffusion models, pushing the state-of-the-art by many steps emperically. - Experiments are thorough in attacking both normal and adversarial trained models - Compared against many baseline unrestricted attacks and proposed attacks outperforms all the methods.

Weaknesses

- The paper lacks technical novelty as the latent space optimisation for generating adversarial attacks as been popular from many years [17]. - Since this is a unrestricted attack, the generated image quality is difficult to assess with metrics. Nonetheless, authors have computed 5 metrics to understand the image quality - Authors did not discuss about the code release to reproduce the experiments. For this particular paper, the implementation of this proposed method is not simple as the section 3.1 and 3.2 mostly discusses about the difficulties in optimising the latent space and how they overcome it through skip gradient, momentum and boundary function.

Questions

Overall, I do not have major questions as the proposed method is properly motivated and leverages the powerful generative model to aid the attack generation. However, I have few minor questions mostly around implementation and design choices: - In section 3.1, authors propose to optimise the null text embedding $∅_t$ at every timestamp to offset the error. Is there any difference to your implementation as compared to the method of [32]. Can you please clarify your contribution in the section 3.1 ie. mapping image $z_0$ to latent space $z_t$. - In particular, authors show the benefit of momentum factor $\mu$ and boundary function $ϱ(·)$ to improve the ASR in Appendix D. Did you perform ablation study to set the value of $\mu$ to 1 and can authors provide more insights on the design of boundary function in particular for inputs outside the valid range [0, 1]? - How do you ensure the perturbation value of $k = 0.1$ in the latent space is not drastically modifying the original image. Can you present an ablation of this key parameter vs attack performance vs image quality ? In Table 3, there is no discussion about the non-reference metrics mentioned in the paper. Why is the method not evaluated with metrics such as LPIPS that captures perceptual similarity? Please also include FID to the metrics. - How is the performance of the attacks in targeted attack setting? Do you require larger shift in latent space to craft example? - Please also benchmark the baseline attacks in terms of attack speed for completeness. - Can you constrain the perturbation to a local region in latent space to generate a kind of patch attacks in image space? Finally, I request the authors to discuss their plans in releasing the codebase to reproduce the experiments as I believe this will be invaluable to the community.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

- The inference time of the attack is much higher taking 2.5 minutes for image due to many many levels of optimisation such as for null text embedding, perturbation in latent space.

Reviewer oMF66/10 · confidence 4/52023-07-14

Summary

This paper proposes the Adversarial Content Attack based on the diffusion model. The proposed attack method first maps the image onto a low-dimensional manifold of natural images and then moves images along the adversarial gradient on the manifold to generate photorealistic adversarial examples. The authors conduct extensive experimentation and demonstrate the efficacy of the Adversarial Content Attack in both normally trained models and defense methods.

Strengths

1. The paper presents a host of visualization and experiments to support the intuitions and conclusions. 2. The investigated topic is important and useful.

Weaknesses

1. This work seems like an application of the diffusion model in model debugging research, finding some hard samples of deep models. However, it lacks mentions or comparisons with related work, including [1],[2], etc. 2. In Table 3, I can hardly understand why the generated adversarial image can achieve better photorealism than the real image. 3. In Table 2, Inc-v3$_{ens4}$ is used as the target model, which is a defense method. But confusingly, the attack success rate (**62.2**) surpasses the case when using normal Inc-v3 (**58.8**) as the target. I will be pleased to raise my score if these questions can be properly answered. [1] Xiaodan Li, Yuefeng Chen, Yao Zhu, Shuhui Wang, Rong Zhang, Hui Xue. ImageNet-E: Benchmarking Neural Network Robustness via Attribute Editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. [2] Maximilian Augustin, Valentyn Boreiko, Francesco Croce, Matthias Hein. Diffusion Visual Counterfactual Explanations. Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021.

Questions

This paper mainly focuses on the untargeted attack. I wonder whether the proposed attack method can also be applied to the targeted attack case, that is, to specify the misclassified category.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The paper has discussed the limitations and negative social impacts of the proposed attack method.

Authorsrebuttal2023-08-15

Happy to Discuss with You

Dear Reviewer frod: As the discussion period is closing, we sincerely look forward to your feedback. We deeply appreciate your valuable time and efforts spent reviewing this paper and helping us improve it. It would be very much appreciated if you could once again help review our responses and let us know if these address or partially address your concerns and if our explanations are heading in the right direction. Please also let us know if there are further questions or comments about this paper. We strive to improve the paper consistently, and it is our pleasure to have your feedback! Best regards, Authors

Reviewer oMF62023-08-15

The authors have addressed my concerns and I appreciate the efforts the authors made to refine the paper. I have raised my score. Though I recommend this paper to be accepted, I am also willing to hear about the other reviewers' further opinions and discussion.

Reviewer frod2023-08-15

I appreciate the authors' efforts in addressing my questions. While a significant portion of my concerns has been satisfactorily addressed, I find that the conclusion drawn regarding the application of the Dunning-Kruger effect in the context of current defense methods remains somewhat unclear to me. It's important to note that the Dunning-Kruger effect is commonly employed to elucidate cognitive biases exhibited by humans. This is also not surprising that the proposed method could be cracked by unseen attacks. Given that the Dunning-Kruger effect has been previously elucidated in [A6], the reviewer acknowledges that this particular concern is minor. I have raised my score. However, the reviewers suggest the authors discuss these previous works in the revision to make the claims more rigorous and substantiated.

Reviewer QZtm2023-08-20

Response to authors

The author answered my questions and I will improve my score.

Reviewer Rhy12023-08-21

Thank you authors for the rebuttal and additional experiments. My concerns regarding momentum $\mu$ and perturbation value $k$ is addressed. The rebuttal experiments suggest that the attack success rate is not overly sensitive to these hyper-parameters. Moreover, I believe the boundary function is an additional contribution which can be extended to other attacks. On the other hand, I request the authors to incorporate the experiments about the attack speed in the revised paper and share an user-friendly reproducible code for the benefit of community. Overall, I believe this paper will inspire the future research on diffusion-based unrestricted attacks and will be of interest to the audience at NeurIPS. I increase my score to WA.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC