CoMat: Aligning Text-to-Image Diffusion Model with Image-to-Text Concept Matching

Diffusion models have demonstrated great success in the field of text-to-image generation. However, alleviating the misalignment between the text prompts and images is still challenging. The root reason behind the misalignment has not been extensively investigated. We observe that the misalignment is caused by inadequate token attention activation. We further attribute this phenomenon to the diffusion model's insufficient condition utilization, which is caused by its training paradigm. To address the issue, we propose CoMat, an end-to-end diffusion model fine-tuning strategy with an image-to-text concept matching mechanism. We leverage an image captioning model to measure image-to-text alignment and guide the diffusion model to revisit ignored tokens. A novel attribute concentration module is also proposed to address the attribute binding problem. Without any image or human preference data, we use only 20K text prompts to fine-tune SDXL to obtain CoMat-SDXL. Extensive experiments show that CoMat-SDXL significantly outperforms the baseline model SDXL in two text-to-image alignment benchmarks and achieves start-of-the-art performance.

Paper

Similar papers

Peer review

Reviewer 9f7Y5/10 · confidence 3/52024-07-10

Summary

The paper introduces CoMat, an end-to-end diffusion model fine-tuning strategy for text-to-image generation that addresses misalignments between text prompts and generated images. This method integrates a novel image-to-text concept activation module and an attribute concentration module, aimed at improving text-to-image alignment.

Strengths

1. **Innovative Integration of Image-to-Text Models**: Utilizing an image-to-text model for concept activation in a diffusion-based approach is a novel application that enhances the alignment between generated images and text prompts. 2. **Comprehensive Modules for Fine-Tuning**: The attribute concentration module and the concept activation module are well-designed to tackle specific challenges in text-to-image generation, such as concept mismapping and concept omission.

Weaknesses

1. **Complexity of Implementation**: The integration of multiple components such as image-to-text models, segmentation models, and the fine-tuning strategy may introduce significant complexity and computational overhead. 2. **Insufficient Comparative Analysis**: Although differences from methods like TokenCompose [1] are discussed, there is a notable lack of direct quantitative comparisons with state-of-the-art methods including TokenCompose [1], Structure Diffusion [2], and Attend-and-Excite [3]. This comparison is crucial to establish the relative performance and advancements over existing techniques. 3. **Resource Intensiveness of Loss Computation**: The loss computation method $L_{i2t}$ may be resource-intensive and inefficient as it requires the diffusion model to undergo multiple denoising steps to generate an image before the loss can be calculated, potentially impacting scalability and practical usage. [1] Wang, Zirui, et al. "TokenCompose: Text-to-Image Diffusion with Token-level Supervision." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024. [2] Feng, Weixi, et al. "Training-free structured diffusion guidance for compositional text-to-image synthesis." arXiv preprint arXiv:2212.05032 (2022). [3] Chefer, Hila, et al. "Attend-and-excite: Attention-based semantic guidance for text-to-image diffusion models." ACM Transactions on Graphics (TOG) 42.4 (2023): 1-10.

Questions

Can the image-to-text model **effectively** offer pixel-level optimization guidance to the diffusion model using $L_{i2t}$? Additionally, are there any ablation studies that demonstrate the effects of using $L_{i2t}$ for training?

Rating

5

Confidence

3

Soundness

2

Presentation

3

Contribution

3

Limitations

The proposed CoMat model incorporates several sophisticated components, including an image-to-text concept activation module and an attribute concentration module. While these additions are innovative, they significantly increase the model's complexity and computational demands. This complexity may limit the scalability of the approach, especially in resource-constrained environments or when handling large-scale datasets.

Reviewer 3go26/10 · confidence 3/52024-07-11

Summary

The paper breaks down the misalignment problem in T2I two: concept ignorance and concept mismapping, and propose a fine-tunning strategy to enhance the prompt understanding and following. The methods include two modules: The concept activation module to maximize the posterior probability; An attribute concentration module is proposed for positive and negative mapping.

Strengths

This paper proposes a general framework for supervising image generation, including a Image to text model scoring method and a prior preservation loss for better text alignment without losing fidelity; and a attribute concentration method which utilizes open vocabulary segmentation method to force attention modification area.

Weaknesses

1. The training cost is disproportionate to this complicated framework. This framework includes diffusion training - full steps inference - mllm judging - grounding segmentation for each entity ... However, the training cost is only ten hours for 8A100 and only needs 2k iterations to converge, and 128 rank lora is enough. I doubt the limited training is enough to convert the SDXL model to attain the attribute learning ability therefore the soundness of this paper. It is needed to provide more training information to testify the experiments are enough and real. 2. The real training cost is a concern since the training involves manipulating diffusion results in the pixel space and inference with MLLM. 3. The proposed method seems cannot assign attributes to multiple same-name objects, such as an Asian girl with an Indian girl, the segmentation model cannot differentiate two girls and therefore cannot assign attributes. Others are questions not weakness, please see the questions part

Questions

1. how to use image-to-text LLVM to provide a matching score?, since most LLVM models are developed for VQA and captioning, scoring is usually not a skill for them. 2. The description of Mixed Latent Strategy is not sufficient. After reading twice, I didn't get the meaning of "in addition to the latent starting from pure noise", T2i training never starts from pure noise but from $x_0 + \epsilon_{t}$, 3. Since the training details concern me, if authors can kindly open-source the code would make the paper's soundness no concern. Would you open-source the code?

Rating

6

Confidence

3

Soundness

2

Presentation

3

Contribution

3

Limitations

Yes, to a certain extent.. The paper doesn't contain a limitation section in the main paper. However, in supplementary material, training cost and mllm usage concern is discussed.

Reviewer 82oN5/10 · confidence 5/52024-07-11

Summary

This paper proposes a fine-tuning strategy for text-to-image diffusion models to improve the alignment of generated images to text prompts. The solution components are summarized as follows: 1. Concept Activation Module: This module helps the model focus on ignored text concepts by leveraging an image-to-text model to supervise the generation process 2. Attribute Concentration Module: This module aims to improve the localization of attributes within images, ensuring that characteristics like color or texture are correctly applied to the right parts of the image.

Strengths

1. An end-to-end fine-tuning strategy is employed to address various text-image misalignment issues in a unified manner, making it easy to deploy.

Weaknesses

1. Functionally, the Fidelity Preservation proposed in this paper is similar to the Class-specific Prior Preservation Loss introduced in reference [1]. Please provide a detailed analysis of the differences between the two. 2. The model presents a potential risk of overfitting the characteristics of the training data, which may affect its generalizability to various real-world scenarios. Therefore, additional experiments related to generalization are necessary. For instance, it should be evaluated whether the model fine-tuned on dataset A can demonstrate improvements when tested on dataset B. 3. The misalignment between generated images and textual concepts is a complex issue with potentially intricate underlying causes. Therefore, it is recommended to use the methods proposed in reference [2] to identify any remaining misalignment issues in the fine-tuned model. [1] Ruiz, Nataniel, et al. "Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023. \ [2] Du, Chengbin, et al. "Stable diffusion is unstable." Advances in Neural Information Processing Systems 36 (2024).

Questions

See weakness.

Rating

5

Confidence

5

Soundness

4

Presentation

4

Contribution

3

Limitations

Already discussed in the paper

Authorsrebuttal2024-08-14

Thanks for your response. We also conduct zero-shot evaluations on the DPG-Bench, which consists of long and complex prompts. Our method exhibits excellent zero-shot performance both qualitatively and quantitatively. Please refer to our rebuttal for Comment 2 for details. We believe this could address the concern of the generalization ability to the long prompts. Besides, we visualize some successful attacks, and we find that the generated prompt contains many meaningless letters, which could largely affect the generation process. An example is: "these small, colorful fiddler crab are known for their distinctively asymmetric claws, with one being much larger than the other. males use their enlarged claw to attract mates and defend their territory., hair, joheat col< troll < shepherds a < ; children "" < lake rest <". Therefore, we argue that the result in Experiment 3 may not fully represent the result of our method's generalization ability.

Reviewer 82oN2024-08-14

There is some controversy regarding the issue of generalization, but this is not a critical flaw. The experimental results with short prompts do show that the model fine-tuning has improved text-image alignment to some extent (although it is difficult to determine the extent of this improvement given the limited amount of data). Therefore, I have decided to raise my score to a Borderline accept.

Reviewer KjKz4/10 · confidence 4/52024-07-14

Summary

This paper studies the prompt-following issues within text-to-image generation models and proposes a very simple yet effective solution by supervising the diffusion models with recognition models like BLIP for image captioning and Grounded-SAM for image segmentation. The authors also propose a fidelity preservation strategy and a mixed latent strategy to address overfitting risks. They fine-tune the SDXL or SD1.5 models on 20K complex text prompts and demonstrate encouraging results on various benchmarks that focus on evaluating attribute binding and object relationship accuracy.

Strengths

- The idea of fine-tuning text-to-image generation models with image recognition models, which act as a reward model, is reasonable and has been explored in previous works such as AlignProp [1] and DRaFT [2]. [1] Aligning Text-to-Image Diffusion Models with Reward Backpropagation, arXiv 2023 [2] Directly Fine-Tuning Diffusion Models on Differentiable Rewards, ICLR 2024 - The experimental results effectively verify the proposed method's effectiveness.

Weaknesses

- The technical novelty is limited, as similar ideas have been explored in both AlignProp [1] and DRaFT [2]. The key difference lies mainly in the different choices of text prompts and the additional supervision on the attention maps. - The authors fail to provide a thorough user study to justify the effectiveness of the proposed approach from the user's perspective. - Although the proposed approach achieves encouraging results, these results focus primarily on fine-grained text-following capability rather than more fundamental challenges such as counting-aware prompts and spatial-aware arrangement prompts. Consequently, the actual contribution of this paper is relatively weak, as the presented results focus on improving benchmark performance rather than addressing the core challenges within text-to-image generation models.

Questions

Refer the Weaknesses

Rating

4

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

Refer the Weaknesses

Reviewer 82oN2024-08-13

Since my concerns have not been fully addressed, I have decided to keep the score unchanged. For example, the author did not provide a positive response to Q3, not even a brief experimental analysis.

Authorsrebuttal2024-08-14

Thank you for your feedback. We sincerely apologize for the concise response and any misunderstanding it may have caused. **We initiated testing of our method using the referenced work [1] at the start of the rebuttal period. However, due to time constraints and limited GPU resources, we were unable to complete the experiments before the rebuttal phase concluded.** Consequently, we plan to present the full extent of our findings in the final draft. Below, we provide details of the experiments conducted and **our current observations** to offer insights into the remaining misalignments we've identified with the assistance of [1]. First of all, we briefly summarize [1]: the paper introduces an attack method to automatically find out what prompt caused Stable Diffusion model to generate misalignment images. Based on the successful attack prompts, [1] summarizes four prompt patterns where Stable Diffusion model often fails to generate aligned images. The four patterns are: a) Variability in Generation Speed; b) Similarity of Coarse-grained Characteristics; c) Polysemy of Words; d) Positioning of Words. Our experiment and analysis contains the following 3 parts: **Experiment 1. Do the discovered four patterns of the original impairment of Stable Diffusion model still exist in CoMat?** Based on the example four patterns in [1], we manually create 20 prompts for the patten (a), (b) and (c), and 5 meta patterns for pattern (d), which generates 30 prompts. Then we use them as prompts for the original Stable Diffusion model and CoMat. The generated images are evaluated by humans. We show the ratio of successful generation below: | Model | Pattern (a) | Pattern (b) | Pattern (c) | Pattern (d) | | --------------------- | ----------- | ----------- | ----------- | ----------- | | Stable Diffusion v1.5 | 45.0% | 35.0% | - | 13.3% | | CoMat-SD1.5 | 65.0% | 50.0% | - | 30.0% | This result align with the enhancements targeted by CoMat. Pattern (a), (b) and (d) all involves multiple objects in the prompt. The Concept Activation Module contributes to the object's existence. Besides, the Attribute Concentration Module restricts the diffusion model to only attend to one object's token for each object area in the image. This further prevents the object combining problem found in pattern (b). The pattern (c) is about the polysemy of words. We argue that without sufficient contextual information, it is infeasible to evaluate the generation result. This experiment demonstrates that CoMat greatly mitigates the original vulnerabilities discovered in Stable Diffusion. However, we acknowledge that there remains a gap between CoMat's performance and perfect alignment, particularly in addressing pattern (c) where prompts contain more than two objects. We are actively working to address these remaining challenges. We apologize that the visualization result cannot be provided in the discussion stage, we will include the visualization result in our final draft. **Experiment 2. Can the learned Gumbel Softmax distribution of Stable Diffusion v1.5 still effectively attack CoMat-SD1.5?** We implement the method of [1] using its open-sourced code in GitHub. Following the setting in [1], we learn a Gumbel Softmax distribution for each class from ImageNet-1K [2]. Due to the time limit, instead of generating 50 images for each class, we generate 5 images for each class and calculate the attack success rate. Then, we directly use this learned Gumble Softmax distribution to attack CoMat-SD1.5. The result is below: | Model | Short Prompt Success | Long Prompt Success | | --------------------- | -------------------- | ------------------- | | Stable Diffusion v1.5 | 47.6% | 51.1% | | CoMat-SD1.5 | 39.8% | 45.3% | As the result shown, CoMat-SD1.5 suffers less from the attack. This result is reasonable since CoMat has gone over the fine-tuning process so the distribution for SD1.5 may be not valid for CoMat-SD1.5. We will include the result following the original setting in [1], where 50 samples are generated for each class, in our final draft. **Experiment 3. How does CoMat act under the auto-attack method proposed in [1]?** Finally, we directly apply the method introduced in [1] to CoMat-SD1.5. Again, due to the time and resource limit, we only test on 5 samples for each class. The result is below: | Model | Short Prompt Success | Long Prompt Success | | ----------- | -------------------- | ------------------- | | CoMat-SD1.5 | 45.2% | 52.3% | Our experiments reveal that CoMat demonstrates superior robustness compared to SD1.5 in scenarios involving short prompt attacks. However, we observed a decline in performance when dealing with long prompts. This discrepancy may be due to the lack of long prompts in CoMat's training dataset.

Authorsrebuttal2024-08-14

Overall, we sincerely apologize for not timely providing the final result of analysis and experiment due to the time and resource limitations. However, we believe the current result delivers insights of the misalignment still present in CoMat. We will cite [1] in the experiment part to discuss these insights in our main paper in the final draft. Hopefully, this can address your concern. We respectfully ask if you would consider reevaluating our work and raising the score. [1] Du, Chengbin, et al. "Stable diffusion is unstable." Advances in Neural Information Processing Systems 36 (2024). [2] Russakovsky, Olga, et al. "Imagenet large scale visual recognition challenge." *International journal of computer vision* 115 (2015): 211-252.

Reviewer 82oN2024-08-14

In Experiment 3, it was shown that after the model was fine-tuned with CoMat, the attack success rate of long prompts actually increased. The authors believe that this discrepancy may be due to the lack of long prompts in CoMat's training dataset. Coupled with the second issue I initially raised, namely the potential risk of overfitting the training data, could this suggest that the method has a weakness in generalization to some extent?

Reviewer 9f7Y2024-08-13

Thank you for your response. Most of my concerns have been addressed. However, I'm still unsure whether the image-to-text model provides effective optimization guidance as presented in Fig. 1 of the author rebuttal PDF. Additionally, I have some concerns about the complexity of the method design. Despite these reservations, I believe this is solid work. Therefore, I’ve decided to raise my score to borderline accept in recognition of your efforts. Thank you for your hard work.

Authorsrebuttal2024-08-13

We sincerely thank the reviewer for the kind support of our work! **A reasonable explanation to effective optimization guidance of the image-to-text model** The image-to-text model is trained to predict the caption, which is highly aligned with the given image. Therefore, once the generated image is not aligned with the prompt, the image-to-text model will be "reluctant" to output the prompt as the caption for the generated image. This "reluctance" comes from the misalignment of certain concepts in the image. The image-to-text model views the image, discovers the misalignment areas, and finally gives a high $\mathcal{L}\_\{i2t}$. We leverage this attribute and try to minimize this "reluctance". The gradient of $\mathcal{L}\_\{i2t}$ mainly summits at the areas where misalignment occurs, as shown in Fig.1 in the author rebuttal PDF. This gradient is further back-propagated to the diffusion model to fix the misalignment of this area. That's where the pixel-level guidance comes from. We will definitely make this point clearer in the revised version!

Reviewer 3go22024-08-13

Thanks for your detailed reply, i'll keep my score as Borderline accept

This paper needs a lot of effort to build the whole pipeline and the result's soundness is credible which makes me maintain the accepted score. On the other hand, the paper's innovation is more about the engineering part with code not open-sourced. So I decided to maintain my score as Borderline accept

Authorsrebuttal2024-08-13

We sincerely thank the reviewer for their support and valuable feedback on our work! We appreciate your suggestions, particularly regarding the clarification of LVLM scoring usage and the mixed latent strategy. These insights will significantly strengthen our paper, and we will incorporate detailed explanations in our revision. Additionally, we will work on to address the issue of assigning attributes to multiple objects with the same name. We believe these revisions above will substantially enhance the quality and contribution of our paper. We are committed to open-sourcing all of our training codes to ensure full reproducibility of the training process upon acceptance. Given this commitment and our planned revisions, we respectfully ask if you would consider reevaluating our work and raising the score.

Reviewer 3go22024-08-13

It's hard for me to raise my score to Weak Accept (although I did at last). CoMat is a complicated framework which needs a lot of engineering techniques if this is an open source project, I think it would be great for the community.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC