HYDRA: Model Factorization Framework for Black-Box LLM Personalization

Summary

The paper proposes HYDRA, a learning-based model factorization framework that captures both user-specific and shared behavior patterns to enable effective personalization within black-box LLMs. The framework involves training a reranker to prioritize the most useful information from top-retrieved relevant historical records, and training an adapter to align the output with individual user-specific preferences, thereby eliminating the reliance on access to inherent model parameters of black-box LLMs. The experimental results demonstrate that HYDRA outperforms existing state-of-the-art prompt-based methods by achieving an average relative improvement of 9.01% across five diverse personalization tasks in the LaMP benchmark.

Strengths

1. The paper proposes a novel method for personalizing black-box LLMs by combining user-specific behavior with shared knowledge, which addresses limitations of previous methods. 2. The experimental results show that HYDRA achieves an average relative improvement of 9.01% over existing state-of-the-art prompt-based methods across five diverse personalization tasks in the LaMP benchmark. 3. The authors plan to release the code repository and model checkpoints, promoting transparency and reproducibility in future research.

Weaknesses

1. The Algorithm Details in Appendix C indicate that HYDRA requires multiple stages of training. However, the paper does not provide an analysis of HYDRA's efficiency. 2. The paper only describes the user-specific head as a single layer of a feed-forward network, without providing details about the base model. 3. HYDRA-Adapter Inference utilizes an LLM to generate b candidates and selects the one with the highest score as the final answer. This approach substantially increases the inference time. 4. As the number of users increases, the number and parameters of user-specific heads also increase accordingly. This not only raises concerns about computational expenses but also introduces storage issues, potentially posing challenges for practical applications. 5. The effectiveness of HYDRA heavily depends on the quality and relevance of historical data, which may not always be available in the real world.

Questions

1. Could you provide the time complexity analysis, training runtime, and inference time? 2. Could you provide a more detailed introduction of the base model? 3. HYDRA-Adapter Inference uses LLM to generate b candidates, then selects the one with the highest score as the final answer. Considering the inherent randomness in LLM generation, is this fair to baselines? 4. How does HYDRA perform in scenarios where there are significant disparities in user behavior history, such as some users having dense histories while others have sparse ones?

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

1. HYDRA-Adapter Inference utilizes an LLM to generate b candidates and selects the one with the highest score as the final answer. This approach substantially increases the inference time. 2. As the number of users increases, the number and parameters of user-specific heads also increase accordingly. This not only raises concerns about computational expenses but also introduces storage issues, potentially posing challenges for practical applications. 3. The effectiveness of HYDRA heavily depends on the quality and relevance of historical data, which may not always be available in the real world.

Summary

This paper provides a black-box LLM personalization framework that explores global and local knowledge from user’s historical behaviour through model factorization.

Strengths

Strengths: 1. The paper is straightforward. The method is reasonable 2. This is good to see the authors provided many details including analyses and case studies in Appendix G and H 3. Extensive experiments were conducted with implementation details and prompts were also provided. 4. Model factorization is a good idea

Weaknesses

Weaknesses: 1. The performance is not convincing. For example, in Table 1, it’s interesting to see that ICL-Random seems to be perform on par with HYDRA (second-best baseline on LaMP-2M, LaMP-5), and even better than RAG / PAG baselines. It’s not clear to me why using random items from user behaviour history is doing much better; please give more details. 2. From the design until experiments, the authors should provide more details (or I may missed it?) about the tasks that benefit by shared (global), the tasks that benefit individual (local) preference, and the tasks that benefit from both. I consider model factorization is a big factor, but not much explanation and analyses were given for that part. In that way, it will further strengthen the paper 3. In addition, in Table 2, for example, HYDRA -P.-Adapter&Reranker has the same performance as HYDRA. Section 4.3 should provide more analyses and explanation about that. 4. Even though the authors provided implementation details, I still believe the source code should be released or more stronger analyses should be given, given the performance results I mentioned above.

Questions

Please refer to my concerns above.

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

The authors did provide details in Appendix A.

Summary

The paper introduces HYDRA, a model factorization framework designed to personalize large language models (LLMs) without modifying their internal parameters. HYDRA addresses the challenge of personalizing inherently opaque, black-box LLMs through a retrieval-augmented workflow. This method enhances personalization by using historical data to effectively capture user-specific preferences. HYDRA consists of two primary components: a personalized reranker that selects relevant information from the user's history, and a personalized adapter that tailors the LLM’s outputs to individual preferences. By integrating both global knowledge and local user behaviors, HYDRA improves personalization effectiveness. Extensive testing shows that HYDRA surpasses other state-of-the-art methods in personalization tasks, proving its ability to deliver tailored experiences without altering the fundamental model parameters.

Strengths

Originality: HYDRA introduces a novel model factorization method to personalize black-box large language models (LLMs). This marks a significant advancement beyond traditional techniques that necessitate access to model parameters. Distinct from existing methods which depend on prompt design or fine-tuning, HYDRA employs a dual-component system comprising a reranker and an adapter. This system enhances personalization without the need to modify the LLM's internal parameters, offering effective personalization within the limitations of black-box models. Clarity: The paper is well-organized and clearly describes the components and operation of HYDRA. The explanations are concise and straightforward. Additionally, the inclusion of figures and detailed descriptions of algorithmic steps offers a clear view of the model’s structure and function, which helps in understanding the model’s mechanics.

Weaknesses

1) Limited Evaluation Metrics: The evaluation primarily emphasizes improvements in accuracy and benchmark performance. However, the use of the LAMP dataset to measure personalization is quite limited and fails to convincingly demonstrate effectiveness. 2) Dependency on High-Quality Data: HYDRA's performance is significantly dependent on the quality of historical user data. If the data is sparse or noisy, the functionality of both the reranker and adapter could be undermined. Exploring and integrating methods to effectively manage such data limitations is crucial. 3) Scalability Concerns: The discussion in the paper about the model's effectiveness covers only a limited number of users. Conducting scalability tests with much larger datasets and more diverse user bases could help in affirmatively validating the framework's efficiency on a larger scale. 4) User Privacy Concerns: HYDRA uses user-specific data, raising potential privacy issues. 5) Handling Dynamically Changing Preferences: How does HYDRA manage continuously evolving user preferences? Is there an efficient mechanism in place to update user models without extensive retraining? 6) Limitations of Reranker and Adapter: What are the constraints on the learning capacity and efficiency of the reranker and adapter modules? Are there situations where these components fail to achieve optimal results? 7) Integration Challenges: What are the main challenges in integrating HYDRA into existing large-scale systems, especially those that already have personalization frameworks in place?

Questions

1) Limited Evaluation Metrics: The evaluation primarily emphasizes improvements in accuracy and benchmark performance. However, the use of the LAMP dataset to measure personalization is quite limited and fails to convincingly demonstrate effectiveness. 2) Dependency on High-Quality Data: HYDRA's performance is significantly dependent on the quality of historical user data. If the data is sparse or noisy, the functionality of both the reranker and adapter could be undermined. Exploring and integrating methods to effectively manage such data limitations is crucial. 3) Scalability Concerns: The discussion in the paper about the model's effectiveness covers only a limited number of users. Conducting scalability tests with much larger datasets and more diverse user bases could help in affirmatively validating the framework's efficiency on a larger scale. 4) User Privacy Concerns: HYDRA uses user-specific data, raising potential privacy issues. 5) Handling Dynamically Changing Preferences: How does HYDRA manage continuously evolving user preferences? Is there an efficient mechanism in place to update user models without extensive retraining? 6) Limitations of Reranker and Adapter: What are the constraints on the learning capacity and efficiency of the reranker and adapter modules? Are there situations where these components fail to achieve optimal results? 7) Integration Challenges: What are the main challenges in integrating HYDRA into existing large-scale systems, especially those that already have personalization frameworks in place?

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

NA

A gentle reminder

Dear Reviewers, Thank you again for your valuable feedback. We would like to kindly remind you that the author/reviewer discussion phase ends soon. We sincerely hope that our responses have enhanced the paper's quality and addressed your concerns. If there are any additional suggestions or comments you would like to provide, please don't hesitate to share them. We look forward to engaging in a constructive discussion during the rebuttal phase. Thank you! Best Regards, Authors