Aligning Vision Models with Human Aesthetics in Retrieval: Benchmarks and Algorithms

Summary

This paper studies the problem of aligning vision models with human aesthetic standards in a retrieval system. There are three key parts in the proposed model including LLM rephrasing, re-ranking, and RL fine-tuning. Two novel benchmarks are also introduced to integrate aesthetic quality into evaluation metrics. Experimental results demonstrate the effectiveness of the proposed method and the benchmarks.

Strengths

1. This paper addresses the aesthetic quality issue in image retrieval systems and introduces a reinforcement learning fine-tuning strategy that enables the retrieval model to directly retrieve images based on both semantics and aesthetic quality, eliminating the need for multi-stage filtering. This approach holds significant value. 2. The paper introduces two evaluation benchmarks, addressing the limitation of current image retrieval benchmarks that fail to evaluate aesthetic quality. 3. The experiments are comprehensive, validating the importance of each component in the proposed method.

Weaknesses

1. The methodological process described in the article is somewhat cumbersome, with Figure 2 merely outlining key processes and concepts in a rudimentary manner, thereby increasing the difficulty for readers to comprehend. 2. The authors appear to conflate "no-reference image quality assessment" with "image aesthetic quality assessment." While these tasks are indeed closely related, they are distinct. MANIQA, for instance, should not be regarded as an aesthetic quality assessment model, and its paper does not evaluate the model's performance on aesthetic datasets. 3. There remain some details in the article that are inadequately explained. It is peculiar that in Appendix Table 7, the same stride seemingly yields a different number of images. 4. The manuscript contains typos. For example, the indicator function symbol in Equation 11 is clearly garbled.

Questions

1. In line 62, it is said that the open-source IAA datasets cannot be used for aesthetic retrieval evaluation. Can you give a further explanation? 2. In the first step of data preprocessing, the authors use the concept of "topic". Can you give a further explanation?

Rating

7

Confidence

5

Soundness

4

Presentation

3

Contribution

3

Limitations

The paper does not discuss the limitations.

Summary

The paper looks into the alignment task for vision and language models within retrieval models where properties such as visual aesthetic comes to play. To achieve this, the paper collects some data to design a metric suitable for taking into account human aesthetic evaluation. And employs an RL-based technique to exploit the human opinion for better aligning the retrieved images with human aesthetic preferences.

Strengths

* It is well-written paper * The concept of aligning vision with aesthetic preferences is interesting and useful in some applications. * The experiments are well-designed and quite convincing. * It is interesting that LLM rephrasing could improve the quality of results

Weaknesses

* The proposed metric could be elaborated better and maybe explained how the study ensured the metric is not designed under influence of the model.

Questions

* The work has been focused on visual aesthetics, given the LLM rephrasing results, it may be beneficial to look into the sophistication of language parts and how that could correlate with the aesthetic of the image. Could that motivate higher level of language sophistication is also correlated with higher visual aesthetics? * Could you elaborate how the RL-based approach could scale?

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The paper does not discuss any limitations with respect to the proposed method.

Summary

This work aims to align vision models with human aesthetic standards in a retrieval system. To do this, the authors propose a preference-based reinforcement learning method that fine-tunes the vision models to distill the knowledge from both LLMs reasoning and the aesthetic models to better align the vision models with human aesthetics. The authors further propose a novel dataset named HPIR to benchmark the alignment with human aesthetics.

Strengths

1. The idea of aligning vision models with human aesthetics in retrieval is interesting. This work has potential applications in various real-life applications. 2. The authors’ motivation of utilizing the reasoning ability of large language models (LLMs) to rephrase the search query and extend the aesthetic expectations is insightful. 3. The paper is well-written and informative. 4. The proposed dataset HPIR can be used by fellow researchers in the related fields.

Weaknesses

I feel it can be further improved in the following ways. 1. For benchmarking human preferences, it might be better to record down the human variance in their annotations. I understand the authors used multiple annotations to ensure robustness, but since aesthetics is a subjective concept, human variance itself tells something. 2. Following point 1, I feel the work can be made more solid if it includes some human evaluation studies on the experimental results. For example, in Fig. 5, it does not seem so obvious to me on the respective enhancement with finetuning. Without the above two points, I feel the paper has somewhat overclaimed the "alighing vision models with human aesthetics".

Questions

1. Have the authors considered human variance in aesthetics perception? 2. Are the objective metrics enough for results evaluation? Have the authors considered using human studies to evaluate the results?

Rating

5

Confidence

2

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors did not indicate limitations in their paper, and mentioned that they will discuss it in future. I feel this paper has clear limitations such as the indication of human variance and the evaluation of the results.

Summary

This paper aligns the vision models with human values by leveraging LLM for query rephrasing and introducing preference-based reinforcement learning. The paper also presents a novel dataset named HPIR to benchmark the alignment with human aesthetics.

Strengths

This paper introduces a novel approach to align visual models with human aesthetics, combining LLM rewriting to enhance query understanding and using preference-based reinforcement learning to fine-tune the model. The paper is comprehensive in experiments and introduces the HPIR dataset for benchmarking. The paper is well-structured and the methods are clearly explained. Key concepts are well defined and the use of diagrams helps to effectively illustrate the results. And this paper improves the aesthetic quality of results in image retrieval by aligning visual models with human preferences. The proposed method and dataset provide valuable ideas for future research in this area.

Weaknesses

[W1] This paper lacks a detailed user study to validate the actual effectiveness of the proposed method. Including a user study with different participants to evaluate the subjective improvement of aesthetic alignment could provide stronger evidence for the actual effectiveness of the method. [W2] Placing the related work in Section 6 makes it difficult for readers to have a clear understanding of the problem domain and existing research results before reading the specific methods and experiments, which is not conducive to the coherence of the paper structure.

Questions

Computational Cost: Can you elaborate on the computational costs and resource requirements of the reinforcement learning-based fine-tuning process? Are there any optimizations that could reduce the computational burden? User Study: Have you conducted any user studies to validate the improvements in aesthetic alignment from a user perspective? If not, do you plan to include such studies in future work?

Rating

6

Confidence

3

Soundness

3

Presentation

2

Contribution

3

Limitations

Exploring possible negative social impacts, such as implications for privacy or how the technology might be misused in unintended ways Research could benefit from deeper analysis of how biases in training data affect model outputs beyond just aesthetics, particularly with respect to cultural and demographic diversity.