GuardT2I: Defending Text-to-Image Models from Adversarial Prompts

Summary

This paper proposes a new defensive method for generative T2I models, termed GuardT2I. GuardT2I utilizes a fine-tuned conditional LLM to map text embedding to explicit prompts and detect the presence of NSFW themes. GuardT2I keeps the target T2I model unchanged thus maintaining the generated image quality. Evaluations are conducted on text-based defenses such OpenAI moderation API.

Strengths

* GuardT2I presents a new way towards adversarial prompt filtering in commercial-level T2I generative models. * Comprehensive evaluations are conducted with detailed implementation setups. * The proposed method is extensible and compatible with any other LLM architectures.

Weaknesses

* In the paper there is a lack of direct comparison between GuardT2I and other types of defenses, such as SafetyChecker (image classifier employed by Stable Diffusion) [1], Safe Latent Diffusion (SLD) [2], and concept removal methods [3]. While I understand some of these defenses may be out of this paper's scope considering that they may rely on the generated image or fine-tuning the target T2I model, it is still necessary to report the gap. * Settings of Table 6 are not clearly explained. The inference time of GuardT2I depends on the length of the recovered prompt. However, SafetyChecker detects NSFW themes from the generated image, which means that the inference time of SafetyChecker will not be influenced by the input prompt. SafetyChecker has fewer parameters than GuardT2I while requiring a longer inference time, which is confusing. Please provide more details related to this experiment. * Selection of evaluated adversarial prompts is not well motivated. There are related adversarial attacks against T2I models such as Ring-A-Bell [4], QF Attack [5], and P4D attack [6], which are not included in this paper. [1] Rando, Javier et al. “Red-Teaming the Stable Diffusion Safety Filter.” ArXiv abs/2210.04610 (2022): n. pag. [2] Schramowski, Patrick, et al. "Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023. [3] Gandikota, Rohit, et al. "Erasing concepts from diffusion models." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023. [4] Tsai, Yu-Lin, et al. "Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?." arXiv preprint arXiv:2310.10012 (2023). [5] Zhuang, Haomin, Yihua Zhang, and Sijia Liu. "A pilot study of query-free adversarial attack against stable diffusion." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023. [6] Chin, Zhi-Yi, et al. "Prompting4debugging: Red-teaming text-to-image diffusion models by finding problematic prompts." arXiv preprint arXiv:2309.06135 (2023). [7] Yang, Yijun, et al. "Mma-diffusion: Multimodal attack on diffusion models." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.

Questions

See weaknesses.

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

NA

Summary

- To defend t2i model from advresarial prompts, this paper presents a novel moderation framework, GuardT2I, that adopts a generative approach to enhance Text-to-Image models’ robustness against adversarial prompts. - Specifically, GuardT2I uses a large language model to conditionally interpret text guidance embeddings for effective detection, avoiding binary classification. - Extensive experiments show GuardT2I significantly outperforms commercial solutions like OpenAI-Moderation and Microsoft Azure Moderator across diverse adversarial scenarios.

Strengths

- The paper is well-written and easy to follow. I like the figures in this paper. - The motivation and idea of this paper are novel, clear and well-explained. - Experiments effectively verify the effectiveness of this work, especially, the evaluation on adaptive attacks.

Weaknesses

- Lacks evaluation and comparison on the standard text-to-image generation task. I am curious about the impact of GuardT2I on benign text prompts.

Questions

- I also want to know if the training dataset for GuardT2I includes adversarial prompts generated by methods like Sneakyp. How is the generalization of the trained LLM ensured?

Rating

7

Confidence

5

Soundness

3

Presentation

3

Contribution

3

Limitations

see weakness

Summary

This paper presents GUARDT2I, a new moderation framework designed to defend against adversarial prompts for text-to-image generation models. Specifically, it uses a large language model to conditionally transform text guidance embeddings into natural language for effective adversarial prompt detection. The experiments demonstrate the effectiveness of GUARDT2I in outperforming commercial detectors.

Strengths

- The paper addresses an important issue by defending against adversarial prompts, which is a timely topic given the increasing popularity and deployment of text-to-image generation models. - The proposed solution of transforming text guidance embeddings into natural language is interesting and well-formulated. - The defense methods proposed have outperformed commercial adversarial prompt detectors in many scenarios. - The evaluations on adaptive attacks are valuable, highlighting the practical importance of the defenses.

Weaknesses

- The introduction and related work sections note that model fine-tuning approaches compromise image quality in normal use cases. Consequently, commercial solutions do not typically use such approaches. The authors acknowledge this, and thus it is important to quantitatively evaluate the impact of the proposed solutions on normal use cases (e.g., image quality) to ensure they do not adversely affect regular prompts, the potential metrics could be FID, etc. - Unlike previous classifier-based approaches, this paper adopts a generator-based approach. Despite achieving better detection performance, it may also introduce a higher delay. I disagree with the claim in Section 5 that the proposed approach does not introduce additional inference time. For each prompt, it inputs into the cLLM and then the verbalizer and the sentence similarity checker, incurring additional inference costs. This could be problematic when adopted by commercial platforms due to the large number of queries per second. - The claim that GUARDT2I outperforms leading commercial solutions by a significant margin should be toned down, as Figures 2 and 5 indicate scenarios where baselines still perform on par with the proposed methods. - In terms of prior attacks, the authors missed a key related work published in 2023 [X]. This should be discussed in the related work section, and the authors should evaluate whether their proposed defense can effectively mitigate it in terms of performance. - A minor concern: the proposed approach may reconstruct NSFW prompts during training, as it feeds unfiltered datasets into cLLM. I wonder how the approach is able to infer the actual meaning of such prompts correctly during inference? [X] Liu, Han, et al. "Riatig: Reliable and imperceptible adversarial text-to-image generation with natural prompts." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.

Questions

- What is the additional latency in running the proposed defenses? - How is the approach able to infer the actual meaning of NSFW prompts correctly during inference, given it learns to reconstruct NSFW prompts during training?

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Please see my above comments.

Summary

The paper introduces a framework designed called GuardT2I to enhance the robustness of Text-to-Image models against adversarial prompts. It allows for the effective detection of adversarial intentions without compromising the performance of the Text-to-Image models.

Strengths

1. This paper is well-written and easy to understand. 2. The experiment results demonstrate the effectiveness of the proposed framework across diverse scenarios. From detecting adversarial prompts to inference time.

Weaknesses

1. The effectiveness of the proposed method heavily relies on the performance of the c·LLM, which could be limited by the quality of its training data. The bias within the c.LLM could also lead to biased interpretations of prompts, like scenes involving cultural background.

Questions

1. What about the performance of GuardT2I on other T2I models like SD2.0, SD2.1, Midjourney, DALLE-3 and so on?

Rating

5

Confidence

4

Soundness

2

Presentation

2

Contribution

2

Limitations

Yes.