Optimizing Prompts for Text-to-Image Generation

Summary

The author proposes a new general framework to optimize prompts for text-to-image generation. Specifically, the framework goes through two stages: a supervised fine-tuning stage on a small sets of human engineered prompts and a reinforcement learning process to explore better prompts. The supervised data were web-crawled from Lexica and is augmented with different methods for diversity. During the reinforcement learning process, the model is optimized with two losses. A relevance loss that is obtained with a frozen CLIP encoder to determine if the image obtained from the modified prompt matches with the original text description; an aesthetic loss that is evaluated against an aesthetic predictor trained on human ratings. Overall, the paper is technically sound and introduces a new framework that can be easily adapted to use.

Strengths

The paper is nicely written and easy to follow. The introduced a two-stage pipeline that trains a LM to modify human prompts for better text-to-image generation. The qualitative and quantitative results are solid.

Weaknesses

The experiment section could use more work to further validate the robustness of the general framework. The choice of the model architecture for image generation (Stable Diffusion V1.4) and the LM (GPT-2) seem a bit arbitrary, especially given that these model were iterated at a very fast pace. Given new iteration on the development of image generation model, it could very well be the case that the limitations are addressed with more advanced image generation model. No experiments were done on the different choices of model architecture.

Questions

In equation (4), the aesthetic score is the averaged difference in the aesthetic predictor score of the modified prompt and the original prompt. I wonder why the image generation from the original prompt is used. What are the expected behaviour when using different choices of image generation model or language model? Could a smaller LM reach the same level of performance / generated quality with a larger image generation model?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

4 excellent

Limitations

As already identified by the authors, the generated results are highly biased towards artistic generation as opposed to photo-realistic images. It would be better if the promptist model could handle a wide range of art styles in generated images. The problem could be mitigated when bigger and better image generation model or language model are used.

Summary

The paper presents a novel framework for automatically optimizing prompts in text-to-image generation models. The authors combine supervised fine-tuning with a pretrained language model and reinforcement learning to explore better prompts. The goal is to generate more aesthetically pleasing images while preserving the original user intentions. Experimental results show that the proposed method outperforms manual prompt engineering in terms of both automatic metrics and human preference ratings. This work contributes significantly to the field of text-to-image generation by providing a systematic way to align user intentions and model-preferred prompts.

Strengths

1. **Originality**: The paper presents a novel framework, "Prompt Adaptation", for automatically optimizing prompts in text-to-image generation models. This approach is innovative as it combines supervised fine-tuning with a pretrained language model and reinforcement learning to explore better prompts. This is a significant contribution to the field of text-to-image generation. 2. **Quality**: The paper is well-structured and provides a clear and detailed description of the proposed framework. The authors have also provided a comprehensive analysis of the results, demonstrating that their method outperforms manual prompt engineering in terms of both automatic metrics and human preference ratings. 3. **Clarity**: The paper is well-written and easy to understand. The authors have done a good job of explaining complex concepts in a clear and concise manner. 4. **Significance**: The proposed framework has the potential to significantly improve the quality of images generated from text prompts, making it a valuable tool for a wide range of applications in areas such as digital art, advertising, and content creation.

Weaknesses

1. **Data Bias**: The authors acknowledge that the prompts used for supervised fine-tuning, which were crawled from the Lexica website, contain biases. For example, they tend to generate more artwork instead of realistic photographs and have a higher proportion of prompts about portraits than other categories. This could limit the generalizability of the proposed framework. 2. **Limited Application**: Currently, the proposed framework is only applied to text-to-image models. While the authors mention plans to extend it to other generative models like text-only models and text-to-video models, this limitation could restrict the current applicability of the framework.

Questions

1. How does the proposed framework handle complex prompts that require the generation of images with multiple objects or scenes? 2. Could the authors provide more details on how the reinforcement learning stage mitigates the biases in the prompts used for supervised fine-tuning?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The main limitation of this study is the bias in the prompts used for supervised fine-tuning. The authors have acknowledged this and suggested that balancing the art styles and objects at the beginning could help address this issue. However, they do not provide a clear strategy for achieving this balance.

Summary

The paper proposes Promptist a prompt engine that leverages a pre-trained LLM (specifically GPT-2) to learn to generate prompts preferred by the model based on original prompt from the user. The approach is simple and very effective. The policy network is first fine-tuned with supervision based on pairs of user inputs and manually engineered prompts, obtained from Lexica.art (90k manually engineered prompts). Synthetic prompts were generated using few different approaches, and then paired with the original engineered prompt from Lexica.art to create paired samples. This initial policy network is used as part of the reward function along with aesthetic and relevance scores. Prompts generated by Promptist performs even better than heavily manually engineered prompts. Experiments section also includes human evaluation comparing against main prompt and manually engineered prompt. Detailed experiments section provides useful insights.

Strengths

S1) Clarity of writing S2) Detailed experiments section providing useful insights S3) The proposed approach produces better results (aesthetics) than heavily manually engineered prompts S4) Simple and very effective approach

Weaknesses

W1) Generalization to other architectures is not clear. Results are shown only for SD 1.4. W2) Generalization to the LM in Promptist is also not clear. Current experiments use GPT-2 W3) CLIPScore of 0.28 seems to be a magic number. See Line 91 and equation 3. It is not clear how to adapt this to new data sets W4) Effectiveness of Promptist for different categories and lengths of prompts is also not clear

Questions

Q1) Would the prompts for the training set need to come from prompts used with SD 1.4+ family? Perhaps just requires the T2I model to be trained based on LAION-2B-en subset?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

4 excellent

Contribution

2 fair

Limitations

Yes

Summary

This paper proposes a method for improving user-provided prompts for text-to-image models to make them more intent-aligned and aesthetically appealing. They use the following method: - Collect a dataset of paired basic and optimized user prompt. - Finetune a LM to take in a basic prompt and produce an optimized prompt. - Define a reward function based on CLIP text-image alignment and aesthetic appeal. (The aesthetic appeal is a new head trained on top of CLIP embeddings using a dataset of human judgements.) - Use PPO with diverse beam search sampling to finetune the prompt-improvement LM to train against this reward. They show that the optimized prompts score better than even human-engineered prompts on aesthetic quality according to human evaluations.

Strengths

Originality: There is plenty of work on finetuning text-to-image models for text-image alignment and aesthetic scores, as well as on designing good hard prompts. I have not seen a paper which specifically optimizes prompts through RL. Quality: The paper's experiments and evaluation seem thorough. Clarity: the paper is well written. Significance: It's a known issue that there are specific keywords which can significantly improve image quality and that casual users of text-to-image models might not know to use these. This seems like a meaningful use case.

Weaknesses

- I wish there were comparisons to other methods of improving text-to-image alignment and aesthetics. The paper does not compare against alternate methods (such as learning soft prompts or finetuning the model directly), claiming these are not preferred because they require you to backprop through the model. They also yield less-interpretable outputs. These are reasonable points, so I think this is a minor weakness. - The related work section also mentions methods for automatically searching for prompts. I'd either like to see comparisons so some of these methods or an explanation of why these comparisons aren't relevant. - There is no analysis on how reliable the CLIP score and the aesthetic score are. (Though the results are backed up by human evaluation.)

Questions

* What does it mean "If the relevance score is relatively reasonable (larger than 0.28), we encourage the model to generate more aesthetically pleasing images". Does this mean you use a binary reward thresholded at .28? * In eqn 3, what are the constants 20 and -5.6 for? These seem arbitrary. (If it's just hyperparameters from a sweep, mention that.) * I wish there a comparison with a heuristic baseline which just includes a handful of common hand-selected tags which can apply across most inputs (e.g. "artstation, hyperdetailed, concept art"). Basically, I'm wondering how well Promptist is able to target the generated tags to the particular user prompt or whether it's has mostly just learned a few common tags which produce aesthetic images of all kinds. Alternatively, a supplement with a couple pages of non-cherrypicked optimized prompts would also answer this for me. * Relatedly, how does the diversity of the styles in Promptist prompts compare to the human-optimized prompts? (i.e. is it possible that human prompts are preferred less because humans ask for a more diverse set of image styles, not all of which score highly on the aesthetic score?)

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

3 good

Limitations

The authors point out the issue that the training dataset is biased towards certain artistic styles. This analysis seems adequate.

Concerns mostly addressed

Thanks for addressing my concerns in the "heuristic baseline" experiment. I will raise my score, though I still think the final version of the paper would benefit from a few additional changes: * In the appendix, include a few pages of non-cherrypicked human prompts and optimized prompts so it's easy to see the differences. * Based on your Q2 response, I think comparisons to methods for automatically searching for prompts would be helpful, though not required.