Boosting Alignment for Post-Unlearning Text-to-Image Generative Models

Additional Questions

We thank the reviewer for those questions. These are great comments and feedback. **Performance of the baseline diffusion model in Table 1.** Thank you for this important point. We've added the baseline SD performance on UA/RA/FID as follows: - **UA (↑):** 0.052 - **RA (↑):** 0.955 - **FID (↓):** 3.815 We will incorporate this result in the revised version. **Robustness of the CLIP alignment score. Is it aligned with human judgment?** This is a very insightful question. We thank the reviewer for this comment. We acknowledge that there might be a gap between the clip alignment score and human judgment. Therefore, we additionally conducted the human evaluation and attached the results here. | | Human Judgment on $D_r$ (↑) | AS on $D_r$ (↑) | |----------|----------------|-----------| | SD | 3.4 | 0.348 | | ESD | 3.0 | 0.330 | | Salun | 1.1 | 0.280 | | RGD (Ours) | 3.2 | 0.352 | For human judgment, we collected responses from 9 subjects who were asked to score a test set. The scoring range was from 1 (least aligned with a given prompt) to 5 (most aligned with a given prompt). Our results demonstrate that our method achieves judgment scores close to those of SD, while Salun performs poorly. The relative ranking on the retain set aligns well with the CLIP alignment score. We will incorporate these results into our revised version. Thank you again for your feedback. **Detailed ablation studies on the LLM in the loop process. Does the number of prompts affect performance?** We thank the reviewer for this great question. We acknowledge that it is important to study the impact of dataset size from “LLM in the loop” and therefore, we have investigated how varying the size of $D_f$ and $D_r$ affects performance. Here are our key findings: Consistency: Across different sizes (400, 800, 1200), our method maintains higher alignment scores. Optimal Size: The size of 800 (reported in our paper) shows the best balance of performance. It matches the setting used in [1], allowing for fair comparison. Smaller Set (400): While still effective, this size shows a slight decrease in alignment scores. This is likely due to simply increased iterations on a smaller dataset. Larger Set (1200): This size can achieve high alignment scores comparable to 800 if we reduce $\alpha$ from 1.5 to 1.15 to balance the increased gradient ascent steps. Therefore, the practical suggestion will be in general, it would be beneficial to include more diverse samples for unlearning to maintain the model utility. **Ablation on the size of $D_f$ and $D_r$ for Nudity Removal.** | AS (↑) | $D_{r, train}$ | $D_{r, test}$ | |------------------------|------------------------------|------------------------------| | SD | 0.357 | 0.352 | | RGD (Ours) \|$D_r$\| = \|$D_f$\| = 400 | 0.336 (0.021) | 0.339 (0.013) | | RGD (Ours) \|$D_r$\| = \|$D_f$\| = 800 | 0.354 (0.003) | 0.350 (0.002) | | RGD (Ours) \|$D_r$\| = \|$D_f$\| = 1200 | 0.352 (0.005) | 0.346 (0.006) | We believe that we have initiated the importance of proper usage of 'LLM in the loop' through this paper, and a more comprehensive study of the design will be valuable for future work. [1] SALUN: EMPOWERING MACHINE UNLEARNING VIA GRADIENT-BASED WEIGHT SALIENCY IN BOTH IMAGE CLASSIFICATION AND GENERATION

Comparison to other baselines

We appreciate the reviewer's thoughtful suggestions regarding baseline comparisons. Our baseline selection was guided by two main principles: **Complementarity:** In inexact unlearning, we can broadly categorize approaches into different categories: 1. **Modifying the parameters:** These methods directly alter the model's weights to remove target knowledge. 2. **Modifying the inference:** These approaches change the inference process without altering the original model. Our work falls into the first category, focusing on modifying the model's parameters. Safe Latent Diffusion (SLD) [1] modifies the inference process to prevent certain concepts from being generated, which falls into the second category. We consider works from the second category as complementary to our approach. Moreover, the method proposed in [2] trains only an adapter and can be applied as a plug-and-play solution to other pre-trained models, which differs from our parameter-level modifications. We believe this also falls into the second category. Therefore, we did not include these methods in our comparisons, as they could potentially be used in conjunction with our method rather than as direct alternatives. However, since [6] falls into the first category, we have added additional comparison results in the general response. **State-of-the-art performance:** We prioritized comparisons with methods that represent the current state-of-the-art in the field. For example, both [1] and [3] have been outperformed by more recent methods such as [4] and [5], which we included in our comparisons. [1] Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models [2] One-dimensional adapter to rule them all: concepts, diffusion models, and erasing applications. [3] Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models [4] Erasing Concepts from Diffusion Models [5] SALUN: EMPOWERING MACHINE UNLEARNING VIA GRADIENT-BASED WEIGHT SALIENCY IN BOTH IMAGE CLASSIFICATION AND GENERATION [6] Ablating Concepts in Text-to-Image Diffusion Models

Some claims are not well substantiated:

**What are the privacy considerations of training on public data?** We thank the reviewer for highlighting this point. Upon reflection, we agree that our statement about privacy concerns in the context of models trained on public data might be too broad for the scope of our paper, given that our work focuses on concept removal instead of privacy. We will revise the paper to calibrate the framing. **Elaboration on how copyright infringements can be addressed by removing concepts from a model.** Thank you for your insightful question regarding the motivation and associated application of our work. Our focus on copyright infringement is motivated by recent developments in the field of generative AI. As described in [1], companies like Stability AI and MidJourney are facing civil lawsuits for training their models on artists' work without consent, enabling their models to replicate the style and work of these artists. In response to this issue, we study "style removal" as a practical application of our unlearning method on the copyright infringement application. For example, we aim to demonstrate that our method can effectively remove a particular artist's style from a model's capabilities. This removal would prevent the model from generating images in the style of the artist whose work was used without permission, thus addressing the copyright infringement issue. [1] On Copyright Risks of Text-to-Image Diffusion Models

Clarity issues relating to the experimental setup:

**UA – how come higher is better on this metric?** We thank the reviewer for pointing out the potential confusion regarding the UA metric. We define UA as 1 - accuracy of the unlearned model on $D_f$ (as noted in line 232 of our paper). This definition aims to measure the model's inability to generate or correctly classify forgotten concepts. Higher UA values indicate better unlearning performance, as they represent a greater error rate on the forget set. In other words, a higher UA suggests that the model has effectively 'forgotten' how to generate or recognize the target concepts. We adopted this metric definition to maintain consistency with previous research [1]. **Rationale of evaluating model utility on the retain set only, not the test set.** Thank you for your question about our evaluation. Our evaluation pipeline for Table 1 focuses on two main aspects: the effectiveness of forgetting the target class and the preservation of model utility for the remaining classes. We don't use a separate test set in the traditional sense but rather evaluate on generated images. *Remaining Accuracy (RA)* is used to evaluate whether the unlearned model can still generate the remaining classes correctly after erasing the target class. We ask the unlearned model to generate images belonging to the remaining classes, which effectively serve as a 'test set'. We calculate the classification performance of these generated images to ensure each class of remaining images is well preserved in the unlearned model. *Unlearning Accuracy (UA)* evaluates how effectively the model has forgotten the target class. We prompt the unlearned model to generate images of the target class and then use a pre-trained classifier to determine if these generated images still belong to the target class. We will make it clear in our revised version by including more details. Thank you again for your question. **Missing experimental details for forget set construction in CIFAR-10 and SD experiments.** Thank you for your great point. We acknowledge the lack of detail regarding the forget set in our initial presentation and will incorporate these details in our revised version. For CIFAR-10 experiments: The forget set $D_f$ comprises all images of a particular target class, which is 5000 images. For Stable Diffusion (SD) experiments: Our forget set has 800 images with the corresponding prompts. The forget prompts are generated by adding the target concept (e.g., "nudity" or specific artist styles) into the retain prompts ($D_r$). **Table 2: How are $D_\text{r, train}$, $D_\text{r, test}$ generated?** We appreciate the reviewer's questions regarding the generation of $D_{r,train}$ and $D_{r,test}$ sets. *For the nudity removal application:* a) We use a structured approach to generate diverse prompts for $D_r$, considering multiple dimensions such as activities, environments, times, and moods provided by a Large Language Model (LLM). b) For each dimension, we use LLMs to suggest multiple subconcepts, incorporating diverse semantics belonging to each dimension such as walking, and sitting in activities. c) To create $D_{r,train}$ and $D_{r,test}$, we split these subconcepts in each dimension into train and test sets, ensuring that there is no overlap between train and test sets. d) We then combine these subconcepts to generate $D_r$. *For the style removal application:* similar to nudity removal, we construct some templates with multiple dimensions such as the artist’s name, actions, environments, and moods, then fill in each dimension with the suggestions from LLMs. Compared between retain set and forget set, the only difference is in the forget set ($D_f$) we use the name of the target that we want to unlearn (e.g., Van Gogh), and use other artists’ names or some virtual names in the retain set ($D_r$). We will revise the paper to incorporate these details, providing a clearer and more comprehensive explanation of our method for generating $D_{r,train}$ and $D_{r,test}$ in each application. **Table 1: How are the metrics computed across the 10 classes?** Thank you for bringing attention to this potentially confusing point. The interpretation provided is correct and reflects our intended meaning. To clarify: the caption "The metrics are averaged across all 10 classes" indeed means that we repeat the unlearning process for each class separately, and the results are averaged to produce the final metrics presented in the table. We will revise the caption to make this clearer in the revised version. **What is I2P?** We thank you for your question for further clarification. I2P stands for "Inappropriate Image Prompts", a collection of prompts designed to evaluate the effectiveness of content moderation in text-to-image models. We have cited this work in line 237 of our paper. [1] SALUN: EMPOWERING MACHINE UNLEARNING VIA GRADIENT-BASED WEIGHT SALIENCY IN BOTH IMAGE CLASSIFICATION AND GENERATION

Additional Questions.

**Explain the proposed approach for diversifying the retain set. In particular, why not generate an initial set of prompts (that aren’t necessarily related to the target concept)?** Thank you for your insightful question about our approach to diversifying the retain set. We fully agree with your suggestion that generating an initial set of prompts unrelated to the target concept would yield more diversity. In fact, this is precisely the approach we take. To clarify: our initial set of retain prompts ($D_r$) is generated to cover a wide range of diverse dimensions unrelated to the target concept we aim to erase. These dimensions include activities, environments, times, moods, and others. At the same time, our retain set aims to preserve the model's understanding of a broader category (e.g., "a person") that could be potentially affected by erasing “nudity.” To create $D_f$, we include target-related concept words (e.g., "nude", "naked") in the diverse prompts generated. For example, a prompt in $D_r$ might be "A person walking in a forest at sunset", while the corresponding $D_f$ prompt would be "A nude person walking in a forest at sunset". This approach ensures no direct relationship between the target concept and the retain concepts, as the initial prompts are generated independently of the target concept. We will incorporate the details of generating a diversified retain set in our revised paper. **How does it relate to the approach usually taken in the literature for constructing the retain set?** We appreciate your question about how our approach relates to existing methods in the literature. The design of the retain set has been largely overlooked in previous studies, despite its crucial role. For example, **ESD** doesn't utilize retain set information, yet claims alignment scores comparable to the original SD based on COCO dataset evaluation. However, our evaluation using "person"-related benign prompts reveals a drop in their alignment scores (as shown in Table 2), likely due to the interconnection between erasing "nudity" and general person image generation. **Salun** uses single repeated prompts for the forget (e.g., "a photo of a nude person") and retain (e.g., "a photo of a person wearing clothes") sets, which can potentially lead to overfitting. Therefore, we believe we have initiated the discussion on the importance of carefully designing the retain set, and our more systematic approach to retain set design opens up interesting avenues for further research. **How different would accuracy results be if using a different pre-trained model? Would the overall conclusions or relative rankings between methods be the same?** Thank you for this important question about the generalizability of our results. To address this, we've conducted additional evaluations using SD v3, the most recent version of the pre-trained model. Our approach and findings are as follows (please refer to the tables in the general response): *Model Architecture:* SD v3 employs a transformer-based architecture (e.g., Diffusion Transformer models) instead of the UNet-based architecture used in previous versions. This significant change allows us to test our method's performance across different model structures. *Model Size:* SD v3 offers a range of model sizes, with the largest being nearly 10 times the size of v1.4. We choose a medium size model with 2B parameters, which is approximately 2 times larger than v1.4. This variability enables us to assess how our method performs across different model capacities. *Evaluation Approach:* We maintained the same hyperparameter settings as in v1.4 to ensure an easy generalization capability. We evaluated two baselines alongside our method, observing their performance under multiple hyperparameter tunings. *Results:* a) Alignment Scores: We observed high alignment scores for both $D_{r,train}$ and $D_{r,test}$ splits with SD v3, effectively mitigating harmful output generation. b) Baseline Comparison: Both baselines showed significant alignment score drops with multiple hyperparameter tunings, and our method continued to outperform them.

Additional Question

**The authors claim that their solution offers a monotonic improvement of each task. Is there any empirical evidence to support this claim?** We appreciate your question regarding empirical evidence for our claim of monotonic improvement. Our claim refers to consistent improvement on both objectives—forgetting the target concept (the 'forget' task) and maintaining performance on retained concepts (the 'retain' task)—without sacrificing one for the other. Theoretically, our claim for monotonic improvement is guaranteed given access to the true loss gradients in Eq. 4, and as the step size tends towards zero. In practice, there are two considerations: 1. The step size cannot be infinitely small and must be balanced against the computational budget of the training algorithm, as smaller step sizes also result in longer training times. 2. The true expected loss (i.e., the expectation of the loss over the entire data distribution) is not tractable, and we must approximate it via the empirical batch-wise loss. This results in minor deviations at each step and a stochastic approximation of the gradients in Eq. 4. To verify our claim in the empirical setting, we perform unlearning with our approach and baseline for the class-wise forgetting experiment, please see Figure 2 in the PDF: - **Forget Losses Comparison:** The left figure shows the loss for the 'forget' task. Our method maintains a higher loss compared to the baseline. This higher loss indicates better 'forgetting' of the target concept, as we want the model to perform poorly on this task. - **Retain Losses Comparison:** The right figure shows the loss for the 'retain' task. Our method maintains a relatively stable and low loss throughout the iterations, especially when compared to the baseline, which shows a sharp increase after 300 iterations. This stability is crucial as it indicates our method consistently preserves model utility on retained concepts. We note that the retain loss doesn't show a decreasing trend because our primary goal for the retain set is to maintain utility, not necessarily to improve it. The unlearning process naturally tends to increase the retain loss as we do gradient ascent. However, our "$\min D_r$" objective counteracts this increase (i.e., decrease the retain loss), resulting in the observed stability.

Official Comment by Authors

Dear Reviewer RWKV, We are happy to hear that our responses have addressed your concerns and questions. We appreciate you taking the time to read our rebuttal and adjust your evaluation accordingly. We will incorporate all the clarifications, additional experimental results, and suggested modifications discussed during the rebuttal into our revised version. Thank you once again for your valuable, constructive feedback and for your consideration. Best regards, Authors