Circumventing Concept Erasure Methods For Text-to-Image Generative Models

Response to Reviewer h1Qn

We thank Reviewer h1Qn for the positive comments and insightful feedback. We are delighted that our paper's structure, clarity, and the significance of the addressed issues have been well-received. In response to your concerns, we offer the following clarifications: **The CI attack method is straightforward. Basically, this paper uses the existing methods, such as Textual Inversion (Gal et al., 2023), for the Concept Inversion Method. The CI methods lack insightful and novel design.** While we mostly utilize Textual Inversion for Concept Inversion, we do think the fact that an existing and straightforward method can circumvent current concept erasure methods emphasizes the vulnerability of post hoc erasure methods. We also want to note that vanilla Textual Inversion does not work for SLD and NP, and we had to invent two novel variants that can search for the appropriate soft embeddings in these methods, where Textual Inversion fails. In particular, section 4.2.3 provides the description of Concept Inversion for NP, and section C.4 in the Appendix contains the pseudocode for Concept Inversion for SLD. **It seems that this paper only shows a number of image cases. The experiments conducted on large-scale datasets are favored.** We have provided several additional qualitative results in the Appendix (Section E). We agree with you that experiments on large-scale datasets can provide a more thorough analysis of current concept erasure methods. Hence, we would like to leave experiments on larger datasets for future work, perhaps as an extended journal version of our work.

Response to Reviewer cgR8

We thank Reviewer cgR8 for the supportive remarks and insightful feedback. We are particularly encouraged by your acknowledgment of the clear logic, comprehensive experimental setup, and the effectiveness of the Concept Inversion method in our work. We would like to address your concerns below: **The authors propose their attack methods under relatively lenient conditions, as evidenced by assumptions (1) and (2) in the PRELIMINARIES section. These assumptions are closer to a white-box setting. However, for most MLaaS (Machine Learning as a Service) platforms, attackers would not have access to the internal details of the model, let alone the ability to add new tokens to its dictionary for executing Concept Inversion attacks. The TRANSFERABILITY method outlined by the authors in Appendix 4.4 also seems to be non-transferable to a black-box environment due to the inability of the attacker to modify the model's internal dictionary. Therefore, the current CI attacks appear to still have limited effectiveness in a black-box setting.** We agree that our attack scenario is closer to the white-box setting. We would like to point out that we want to use Concept Inversion attacks to better understand current erasure methods through their failure cases. Our results on Concept Inversion and the transferability of the learned embeddings demonstrate that current concept erasure methods are not fully excising the targeted concept but rather performing some input filtering. We have updated the main draft to clarify this better.

Response to Reviewer YpMN

We are grateful to Reviewer YpMN for their positive and insightful assessment of our work. Your recognition of the novelty, clarity, and the comprehensive nature of our experiments is highly appreciated. We would like to address your concerns to further clarify the aspects of our research below: **There is curiosity about the behavior of the learned concept placeholder embedding in comparison to the embedding of the original concept name. This comparison could provide valuable insights into the effectiveness of the proposed approach.** We agree this is an important question. While optimizing for concept placeholder embedding which is very close to the initial concept does result in deteriorated performance, the placeholder embeddings further away from the original word behave as well within a sentence as the original word embeddings (Please see Figure 7). We leave further investigation of the properties with these embeddings for future work. **Typo: inconsistency in the caption of Figure 7, which does not align with the order of the figure.** We have updated the caption to better align with the figure. **How to understand ‘remapping the concept in token space’? Since the original text encoder and the embedding of the original text tokens are not updated, right?** You are absolutely correct that the original text encoder and the embedding of the original text tokens are not updated. What we are trying to convey is that current concept erasure methods are suppressing the generation of the targeted concept when conditioned on the input embeddings corresponding to the concept text, i.e. Van Gogh. However, there exist word embeddings that trigger the generation of the erased concept in SD 1.4 that also work on the erased model. This means that some degree of input-filtering is occurring. Moreover, since the behavior of the post-erasure model is changed due to fine-tuning the UNet or interference of the generation process, we believe that the erased concept is also mapped to some input embeddings that do not necessarily transfer to the pre-erasure model. We will add further clarifications in the revised manuscript. **how the authors conducted experiments to demonstrate the editability, as the left figure of Figure 7 does not appear to clearly showcase this aspect?** Following Gal et al. 2022, we conducted the editability experiment by first performing Concept Inversion on the erased model, and then generating images using prompts containing the special learned token (e.g. “A painting of the beach in the style of <special-token>"). We then measure the CLIP [1] similarity between the generated images and the text prompt without the special token (e.g. “A painting of the beach”). This allows us to evaluate whether the learned embeddings can be used in various contexts. *__Reference:__* [1] Radford, Alec, et al. "Learning transferable visual models from natural language supervision." International Conference on Machine Learning, 2021.

Response to Reviewer PvZ8

We thank Reviewer PvZ8 for the constructive comments and insightful feedback. We are pleased to know that the reviewer finds our paper clear and well-structured, and our experiments rigorous. We would like to address the concerns below: **The paper evaluates black-box methods using a white-box approach. For example, Kumari et al. [1] already acknowledged their limitation in the white-box setting.** We fully acknowledge the transparency by Kumari et al. [1] in the white-box setting. Nevertheless, this limitation has not been clearly stated in the remaining concept erasure methods. Moreover, while Kumari et al. mention that their method does not prevent the re-introduction of the erased concept in the white-box setting, we are the first to point out that it is their method (and 6 other concept erasure methods covered in our work) mostly provide no protection against more sophisticated inputs. **Clarification is needed regarding the attack scenarios presented. Specifically, it is unclear why an adversary with white-box access would seek to bypass a black-box model.** Through an adversary with white-box access, our work aims to understand better the failure cases of concept erasure methods. Our experiments reveal that the seemingly erased concepts still persist within the model post-erasure. Hence, a malicious user can exploit this false sense of security and trigger the erased model to generate sensitive content. **Assumption 5 in section 3 appears inconsistent with a white-box setting, particularly when considering Assumption 4, which grants the adversary computational resources. It raises the question of why an adversary could not alter the weights of the "erased" model under these conditions.** We would like to clarify that we are not merely trying to generate erased concepts from sanitized models. From a scientific perspective, our goal in this paper is to investigate two questions: (1) does the erased concept still persist in the model post-erasure? (2) are current concept erasure methods performing input filtering? While fine-tuning the entire model on samples containing the erased concept can re-introduce that concept back into the model, it does not help us answer the previously mentioned questions. Hence, we want to restrict the adversary from altering the weights of the erased model. **Regarding the practicality of the proposed attack method, it is important to note that widely-used public services such as Stability AI and DALL-E 3 do not currently accept text-embedding inputs to my knowledge. This may render the evaluation less critical, especially considering that existing methods effectively erase explicitly mentioned words.** We agree with the reviewer regarding the practicality of our proposed attack method towards closed models which are available only through API access. However, the main goal of our work is to take a step toward understanding current concept erasure methods and their vulnerabilities. We hope that our work can shed light on the limitations of these methods to the research community, which can encourage future works to focus more on practical attacks and appropriate defenses. Moreover, our experiments with white-box access demonstrate that (1) current concept erasure methods are susceptible to soft-embedding attacks, and (2) there exists soft-embeddings that can trigger both the original (unerased) model and the fine-tuned (erased) model to generate the target concepts. One possible attack under the black-box assumption is exhaustively performing discrete optimization (black-box) attacks on the unerased model until we find the hard prompts that “jailbreak” the erased model. We leave further explorations of black-box attacks for future work. *__Reference:__* [1] Kumari et al., “Ablating Concepts in Text-to-Image Diffusion Models”, International Conference on Computer Vision, 2023.

Response to Reviewer iwRd

We sincerely thank Reviewer iwRd for their constructive comments and valuable feedback on our submission. We are gratified that the reviewer acknowledges the importance of the problem our paper addresses in the context of text-to-image generation models, and appreciates the clarity of our visual illustrations, the comprehensive nature of our results, and the detailed appendix provided. Below, we address the concerns raised to further clarify our work: **Only 1 version of Stable Diffusion (version 1.4) is presented. Is there any reason to choose this? Will the results be similar in later versions of Stable Diffusion?** The reason why we only consider SD 1.4 is because nearly all results of 7 concept erasure methods used SD 1.4. We have provided additional results of applying AC [1] and SLD [2] on SD 2.0 as well as performing Concept Inversion on the corresponding erased models. Please see Section F of the Appendix in the revised manuscript. **Though the paper exposes that the seven recently proposed concept erasure methods can be broken, the paper does not address in detail how these can be fixed. A detailed discussion section on this will be useful and make the paper stronger.** We have added a discussion section in the Appendix on potential solutions to make concept erasure methods more effective and robust against adversarial inputs. Please see Section A of the Appendix in the revised manuscript for the discussion. *__Reference:__* [1] Kumari et al., “Ablating Concepts in Text-to-Image Diffusion Models”, International Conference on Computer Vision, 2023. [2] Schramowski, Patrick, et al. "Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models.", Conference on Computer Vision and Pattern Recognition, 2023.