Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Models

Summary

This paper introduces AdvUnlearn, a robust unlearning framework that integrates adversarial training into diffusion models to enhance concept erasure. This method aims to prevent DMs from generating harmful or inappropriate content, such as nudity, even under adversarial prompt attacks. AdvUnlearn focuses on optimizing the text encoder rather than the UNet, achieving a balance between effective concept erasure and high image generation quality. Extensive experiments demonstrate AdvUnlearn's robustness and utility across various unlearning scenarios.

Strengths

1. Enhanced Robustness: AdvUnlearn improves the robustness of diffusion models against adversarial prompt attacks, effectively preventing the generation of undesired content. 2. Maintains Image Quality: By focusing on the text encoder and utilizing utility-retaining regularization, AdvUnlearn preserves high image generation quality post-unlearning. 3. Plug-and-Play Usability: The method allows the robust text encoder to be shared across different DMs, making it easy to implement and enhancing its practical usability.

Weaknesses

1. ASR needs to be measured based on more baseline attacks. Since the proposed method is based on the CLIP text encoder, incorporating CLIP-based adversarial attack methods [1] and [2] would provide greater understanding. [1] Wen et al., Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery\ [2] Mahajan et al., Prompting Hard or Hardly Prompting: Prompt Inversion for Text-to-Image Diffusion Models

Questions

1. I thoroughly reviewed the algorithm in the Appendix. However, I need more details on how the text encoder is optimized using Eq.(5). The current form of Eq.(5) does not seem to include the text encoder. Please provide more details on this process.

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

1. More discussion comparing the proposed method with CLIP-based attacks [1] and [2] would be welcomed. [1] Wen et al., "Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery"\ [2] Mahajan et al., "Prompting Hard or Hardly Prompting: Prompt Inversion for Text-to-Image Diffusion Models"

Summary

The authors propose a method for robust (against adversarial attacks) concept-erasing for latent diffusion models. Specifically text-to-image diffusion models. The main contributions are: 1. Integrating adversarial training into machine unlearning by modifying it as a bi-level optimization problem. 2. Contrary to the conventional techniques where only the parameters of the UNeT is updated, the authors show that updating the text encoder can effectively maintains a robustness-utility trade-off.

Strengths

1. The problem of robust-concept erasing is relatively new and important to handle. 2. The authors have formulated the task clearly and proposed a solution to address this important problem. 3. The initial results on text-based attacks (UnlearnDiffAttack) demonstrate an interesting direction.

Weaknesses

Lack of Sufficient Evaluation: 1. The authors majorly evaluate the robustness of the method against only the UnlearnDiff attack, which is a text-based attack. 2. They have not extended their evaluations to other recent peer-reviewed SOTA attacks such as the CCE (ICLR-24) [1] and RAB (ICLR-24) [2] attacks. 3. Specifically, CCE is a strong inversion-based attack that introduces a new token into the vocabulary. CCE relatively takes much less time than the UnlearnDiffAttack. 4. GIVEN CCE ATTACK INVERTS THE ERASED CONCEPT INTO A NEW TOKEN, THE AUTHOR'S CLAIMS OF UPDATING ONLY THE TEXT-ENCODER FOR A BETTER ROBUSTNESS-UTILITY TRADEOFF CAN BE CALLED INTO QUESTION. 5. In addition to the UnlearnDiffAttack, the performance of AdvUnlearn on CCE would be the right benchmark to assess the model's robustness. Lack of Completeness: 1. The authors have not presented the quantitative erasing results of AdvUnlearn for any of the unlearning scenarios. In addition to the robustness and utility performance, studying the erasing performance of the method is important to understand whether optimizing for the former two properties has any compromise on the latter's performance. References: [1] https://arxiv.org/pdf/2308.01508 (Circumventing Concept Erasure Methods For Text-to-Image Generative Models) [2] https://arxiv.org/pdf/2310.10012 (RING-A-BELL! HOW RELIABLE ARE CONCEPT REMOVAL METHODS FOR DIFFUSION MODELS?)

Questions

Nudity Unlearning: 1. Why have the authors chosen only a subset of the 4703 I2P prompts? 2. How many prompts are there in the subset? 2. On what filters was the subset chosen? 3. Please elaborate on why the erasing results on the original I2P benchmark and the nudity count are missing. Style Unlearning: 1. Please elaborate on why the erasing results for the art unlearning scenario is not discussed. 2. The authors have not discussed how erasing the style of one artist affects the other artistic styles. Objects Unlearning: 1. Please elaborate on why the erasing results of each of the object scenarios are not discussed. 2. For the object unlearning scenarios, the experimental setup of evaluating against only 50 prompts seems limited. Baseline method such as ESD evaluate on a large set of 500 samples and also present the results on the remaining classes to understand the loss of utility. This analysis is missing here. Utility: 1. Why have the authors chosen to compute the FID and CLIP scores on the subset of 10K prompts and not the standard 30K prompts like previous methods in the benchmark? 2. Why do the FID and CLIP scores of AdvUnlearn vary significantly for different concept/unlearning scenarios? eg: FID/CLIP score is 19.34/0.29 for nudity-unlearning and 16.96/0.30 for style-unlearning? On the other hand, the performance of baseline ESD does not vary as significantly: 18.18/0.30 (nudity) vs 18.71/0.30 (style).

Rating

5

Confidence

5

Soundness

2

Presentation

3

Contribution

3

Limitations

1. The authors have discussed the limitations of the proposed method. 2. However, as highlighted in the "weakness" section, the author's claim of robustification through updating only the text-encoder cannot be verified unless evaluated against other adversarial attacks from the literature such as CCE and RAB.

Summary

This paper proposes AdvUnlearn, a method aimed at enhancing the robustness of concept erasing. The approach integrates the principles of adversarial training into the unlearning process. Specifically, the text encoder is identified as the most suitable module for robustification. Experiments are conducted on eight DM unlearning methods, including nudity unlearning, style unlearning, and object unlearning.

Strengths

1, The problem addressed in this paper is significant. Enhancing the robustness of concept erasing could substantially reduce safety risks and copyright violations. 2, The idea is intuitive and reasonable. Adversarial training can be used to enhance robustness, and the paper identifies a straightforward method to preserve image quality while focusing on the text encoder as the most suitable module for robustification. 3, The experiments are relatively thorough, including eight unlearning baselines and various concept-erasing tasks.

Weaknesses

The experimental section lacks a sufficient number of attack methods. The introduction to C_{retrain} is inadequate.

Questions

The key questions preventing me from increasing my rating are: I don't understand why only UnlearnDiffAtk is used for robustness evaluation. Since the robustification targets the text encoder, it seems that the textual inversion-based attack [65] should also be used for robustness evaluation. Additionally, I suggest that more reasonable attacks be considered for robustness evaluation. I also question the approach of mainly robustifying the text encoder. Even after adversarial training, I believe there will still be an embedding that can generate the target concept. This defense could be easily bypassed by some embedding-based attack. I don't understand how C_{retrain} is selected to retain model utility. Are the concepts not covered in C_{retrain} still suffering from degradation?

Rating

5

Confidence

2

Soundness

3

Presentation

3

Contribution

3

Limitations

The limitations have been discussed in the appendix.

Similar idea

Dear authors, Congrats on the great work! Interestingly, our work (also accepted to NeurIPS 2024) shares a similar adversarial training idea for concept erasure. So I’d like to take this chance to promote our work :D and help the community better understand the differences between the two approaches. :D We have included a comparison with AdvUnlearn in our arXiv paper, and would greatly appreciate it if the authors could consider updating your manuscript to reflect the relation between the two works. **About our work**: In our work, we propose an adversarial framework for concept erasure that formulates the problem as a bilevel min–max optimization. The core idea is to identify an adversarial concept that, when preserved, makes it more difficult for the model to retain the unwanted concept. We then fine-tune the UNet to erase the target concept while preserving the adversarial one—thereby improving both erasure quality and robustness. Rather than relying on a predefined retained set, our method adaptively selects what to preserve using a learned optimization strategy. We employ the Gumbel-Softmax trick to handle the discrete nature of concept selection, allowing us to optimize over token-based prompts in a differentiable way. **Key differences from AdvUnlearn:**: **Motivation**: AdvUnlearn is inspired by adversarial prompt attacks; our approach is driven by the empirical observation that the choice of preserved concept strongly affects erasure quality and model performance. **Optimization Target**: AdvUnlearn fine-tunes the text encoder, while we fine-tune the UNet, directly influencing the image generation pipeline. **Preservation Strategy**: AdvUnlearn requires a fixed retained set; our method adaptively selects the concept to preserve via Gumbel-Softmax. **Optimization Approach**: We use Gumbel-Softmax, whereas AdvUnlearn uses FGSM for prompt search. Link to the project page with code: https://tuananhbui89.github.io/projects/adversarial-preservation/