UniFL: Improve Latent Diffusion Model via Unified Feedback Learning

Summary

This paper introduces UniFL, a novel approach for improving diffusion models through unified feedback learning. The objective of UniFL is to enhance visual generation quality, preference aesthetics, and inference acceleration. To achieve this, the paper proposes three key components: perceptual feedback learning, decoupled aesthetic feedback learning, and adversarial feedback learning. UniFL is designed to be a two-stage training pipeline that can be applied to various models and yields impressive improvements in both generation quality and acceleration. The paper provides experimental results demonstrating the effectiveness of UniFL in terms of generation quality and acceleration. Additionally, the paper discusses the potential broader impacts and ethical implications of advancements in image generation techniques.

Strengths

1. The writing of this article is commendable, with a well-structured format. 2. The author conducted a large number of visualization experiments to illustrate the focus and effectiveness of the method, which is very attractive. 3. The proposed method serves as a plug-and-play indeed improves the performance on both SD15 and SDXL.

Weaknesses

1. There are some minor LaTeX formatting issues: for example, there should be a consistent space before parentheses in abbreviations, the quotes in the Appendix should be implemented using `’, and “our” should be “Our” in line 130. 2. The values of hyper-parameters such as \alpha_d are not explicitly stated in the paper.

Questions

1. Have the authors explored the impact of different aesthetics on generated images? 2. Are the user studies enough to prove the effectiveness of each module in this field? 3. Why not compare with SDXL-IR and SDXL-DPO in Ablation on Acceleration Steps?

Rating

6

Confidence

2

Soundness

3

Presentation

3

Contribution

3

Limitations

Yes.

Summary

Considering that current diffusion models still suffer from several limitations, this paper aims to propose a unified framework, UniFL to address the main existing challenges by applying feedback learning. To respectively solve the issues of visual distortion, poor aesthetic appeal, and inefficient inference, this paper demonstrates different sub-modules, namely Perceptual Feedback Learning, Decoupled Feedback Learning, and Adversarial Feedback Learning. By fully leveraging the ability of Perceptual Feedback Learning to fine-tune the diffusion during the two training stages, the proposed model aims to achieve both remarkable generation quality and speed simultaneously. Sufficient experiments, including extensive ablation studies for different proposed sub-modules, have proved the validation and effectiveness of the model’s design.

Strengths

The proposed idea about a combination of feedback learning and diffusion models is well introduced with necessary background information and is sufficiently motivated. The three main issues remaining for diffusion, namely inferior quality, lack of aesthetics, and slow inference speed are well addressed, keeping the following methodology sections’ logic pipeline clear and understandable. The proof of the methodology part is adequately explained with abundant support of pipeline figures and pseudocode that help the reader get full access to the novelty insight. Sufficient experiments over different domains, especially the wide range of ablation studies, have been carried out validly which demonstrate the effectiveness of all the proposed feedback learning modules, which enhances persuasiveness.

Weaknesses

The proposed qualitative visualization comparison mainly focuses on justifying the overall generation style and the structure correctness. However, a main issue that may occur during the inference speed-up is detail loss. It will be more appreciated if the author can provide some further visual comparison results to demonstrate the model’s ability to keep the visualization results consistent with that adjective in the text prompt context. Also, according to Figure 7, an unwelcomed issue can be found that the images generated by UniFL for text prompt A cream-colored labradoodle wearing glasses and a black beret teaching calculus at a blackboard mistakenly wears a black tie, which has never been mentioned in the text prompt. It shows an unexpected trend of overfitting and inconsistency. Further explanation about the cause of such results should be addressed.

Questions

Please see the weakness part.

Rating

6

Confidence

5

Soundness

3

Presentation

3

Contribution

3

Limitations

Though there are still some blemishes part for some of the proposed results, generally the experiment part is well-organized, and filled with abundant ablation studies for the proposed modules.

Summary

The work introduces Unified Feedback Learning (UniFL), a unified framework to enhance diffusion models through feedback learning. It addresses three main challenges in diffusion models: visual quality, aesthetic appeal, and inference efficiency. UniFL comprises perceptual feedback learning, decoupled feedback learning, and adversarial feedback learning. - Perceptual Feedback Learning utilizes existing perceptual models, such as an instance segmentation model, to fine-tune diffusion models on a specific aspect. - Decoupled Feedback Learning decomposes the general aesthetic concept into color, layout, lighting, and detail. It fine-tunes the model by reward feedback learning in all these sub-categories. - Adversarial Feedback Learning exploits a general reward model as a discriminator to improve the generation quality of fewer denoising steps. Experiments demonstrate that UniFL improves the performance of diffusion models like SD1.5 and SDXL in terms of generation quality, aesthetic preference, and inference speed.

Strengths

- The paper is clear and easy to follow - The proposed method utilizes different priors from other perceptual models to improve diffusion models. - New aesthetic human feedback dataset is proposed.

Weaknesses

- The overall method is some modification from ReFL. - The perceptual feedback learning part is limited by the selected models, which seem to only fine-tune concepts used to train these perceptual models. - The performance is not significantly superior to other methods according to Table 1.

Questions

- If users want to generate images containing concepts not shown in COCO dataset, but shown in LAION, it seems perceptual feedback learning part will not work. - If users want to fine-tune the diffusion model in several different perceptual aspects simultaneously, will the priors from different perceptual models interrupt each other? - Weaknesses above

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

Yes, the authors addressed the limitations.

Summary

This paper proposes a framework to enhance visual quality, aesthetic appeal, and inference efficiency using various methods, including perceptual feedback learning, decoupled feedback learning, and adversarial feedback learning. Good experimental results are observed.

Strengths

The concept of perceptual feedback learning is promising, as it can leverage various pretrained expert models to improve learning. The idea of decoupled feedback learning makes sense, as it allows the model to focus on fine-grained details in the images.

Weaknesses

The author claims that previous works primarily focus on individual problems through specialized designs and proposes a unified framework. However, three different methods are designed to solve different problems. Why is it called unified? In Line 145, it states that image content is incorporated as an additional condition for guidance. However, as shown in Figure 1, no extra inputs are added to the diffusion model. In the experiment, only an instance segmentation model is used for perceptual feedback learning. Have the authors tried other types of models? It would be interesting to see some analysis on this aspect. For decoupled feedback learning in Equation 5, why is a hinge loss used instead of a winner-loser loss as in Equation 4? What do the data annotations look like? How to choose the hyper-parameters of the hinge loss? What are the details of the semantic-based prompt filter and nearest neighbor prompt compression described in Line 189 for active prompt selection? In Line 202, it states that samples with low inference steps tend to be too noisy to obtain correct rewarding scores in previous methods. How does the proposed adversarial feedback learning improve this and how does it accelerate inference?

Questions

see weakness

Rating

4

Confidence

4

Soundness

2

Presentation

2

Contribution

2

Limitations

see weakness

Kindly reminder to respond to author responses

Dear Reviewers, Thank you very much again for performing this extremely valuable service to the NeurIPS authors and organizers. As the authors have provided detailed responses, it would be great if you could check them and see if your concerns have been addressed. Your prompt feedback would provide an opportunity for the authors to offer additional clarifications if needed. Cheers, AC