Adaptive Preference Scaling for Reinforcement Learning with Human Feedback

Reinforcement learning from human feedback (RLHF) is a prevalent approach to align AI systems with human values by learning rewards from human preference data. Due to various reasons, however, such data typically takes the form of rankings over pairs of trajectory segments, which fails to capture the varying strengths of preferences across different pairs. In this paper, we propose a novel adaptive preference loss, underpinned by distributionally robust optimization (DRO), designed to address this uncertainty in preference strength. By incorporating an adaptive scaling parameter into the loss for each pair, our method increases the flexibility of the reward function. Specifically, it assigns small scaling parameters to pairs with ambiguous preferences, leading to more comparable rewards, and large scaling parameters to those with clear preferences for more distinct rewards. Computationally, our proposed loss function is strictly convex and univariate with respect to each scaling parameter, enabling its efficient optimization through a simple second-order algorithm. Our method is versatile and can be readily adapted to various preference optimization frameworks, including direct preference optimization (DPO). Our experiments with robotic control and natural language generation with large language models (LLMs) show that our method not only improves policy performance but also aligns reward function selection more closely with policy optimization, simplifying the hyperparameter tuning process.

Paper

Similar papers

Peer review

Reviewer 4UPH6/10 · confidence 4/52024-06-20

Summary

To learn a versatile rewards essential for the downstream policy optimization, this paper introduces a novel adaptive preference loss function inspired by distributionally robust optimization (DRO). The proposed approach incorporates an learnable instance-specific scaling factor to accommodate varying uncertainties of preference strength and change the scaling between the preference distribution and reward difference to be non-linear. The proposed loss function for learning the scaling parameters is strictly convex, and thus they only induce negligible additional computational overhead. Experiment results on robotic control tasks and LLMs verify the proposed method.

Strengths

1. Adapting the optimization strength by factoring in the characteristics of each preference pair is an interesting direction. 2. The proposed method is well (theoretically) motivated. 3. The proposed method work well in practice.

Weaknesses

1. The adaptive scaling is defined on a per-instance basis, which necessitates significant compute costs and hinders real-world mini-batch learning. This is evident from Appendix, where the algorithm box shows that the training process operates in a per-datapoint manner 2. Some of the derivation may be simplified, both to make the paper more succinct and leave room for algorithm box as well as other technical details. 3. L221: could you theoretically justify the incorporation of adaptive preference scaling into DPO. e.g., in the framework of KL-control as in the original DPO paper?

Questions

1. L71 "Prior work on this topic is very limited.": There are actually a vast pool of literature concerning loss functions for (RLHF's) reward modeling, a quick pointer is [1] for language tasks and [2] for image generation, as well as the literature review in them. I encourage the authors to add proper discussions and citations to make the claims more precise. 2. L140-141: Could you elaborate this sentence more: "For some applications, it may lead to a reward function that is not flexible enough to differentiate a pair of segments"? In particular, why is the resulted reward function not flexible enough? And how is this related to the linear scaling? 3. In Eq. (3), why do we need to have $KL(p, 1/2)$ in both the objective function and constraint? *** [1] Yang, Shentao, et al. "Preference-grounded token-level guidance for language model fine-tuning." Advances in Neural Information Processing Systems 36 (2023). [2] Yang, Shentao, Tianqi Chen, and Mingyuan Zhou. "A Dense Reward View on Aligning Text-to-Image Diffusion with Preference." Forty-first International Conference on Machine Learning. 2024.

Rating

6

Confidence

4

Soundness

3

Presentation

2

Contribution

2

Limitations

A discussion on limitations and/or potential negative societal impact seems missing.

Reviewer ZDuk5/10 · confidence 4/52024-07-12

Summary

This paper studies the problem of learning from preference data and introduces a learnable scaling parameter for each preference sample. The authors propose an adaptive preference loss function that assign small scaling parameters to ambiguous preferences pairs and large scaling parameters to clear preferences. Experiments demonstrate the improvement of policy optimization performance and efficient of the hyperparameter tuning.

Strengths

Considering different samples have different preference strength, using an adaptive preference scaling makes sense for preference learning. Experiments are conducted on both robotic control and natural language generation tasks.

Weaknesses

The increase in flexibility of the proposed reward function is not verified. The authors claim that one of the limitations of the BT model is the logit of the preference distribution scales linearly with the reward difference. However, according to Proposition 3.1, the limitation still exists. There is no empirical result that supports the claim. Missing the ablation study of the regularization term in the proposed loss function. Missing the RLHF baseline in the natural language generation task. “Ada-Pref demonstrates greater resistance to performance degradation than Pref, indicating its superior ability to align the learned reward function with policy optimization.” It seems that it is not true in the Hopper task? Minors: Line 120: The notation of state transition function is conflicted with preference distribution. Please clarify the meaning of M and K in Algorithm 1.

Questions

I do not believe the proposed loss function is convex, can you provide the proof of Remark 3.1? Why the reward models trained with the proposed method can better guide policy model selection considering the reward model performance is not improved? How do the authors obtain the true reward difference in Section 4.3?

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The computation overhead introduced by learning the scaling parameter is not discussed.

Reviewer ZDuk2024-08-13

I appreciate the authors' detailed reply. They have addressed most of my concerns. However, there is no evidence (even without a toy example) demonstrating that the BT model would fail due to its linear relationship. This makes it unclear for researchers to determine when the proposed method, which seems hard to re-implement, should be used. Therefore, I would keep my previous evaluation.

Authorsrebuttal2024-08-13

Thank you for your insightful comment. We would like to clarify that the BT model does not completely fail but is indeed insufficient in some cases. For example, when the relationship between the ground truth reward difference and the logit is non-linear, the BT model cannot fully capture and learn this correspondence. In our synthetic case, we only know the ground truth reward, not the logit of the preference distribution, so we can't provide direct evidence even in a toy example. However, the superior performance of our method across various tasks suggests that non-linear relationships do exist in real data. While the overall performance of the BT model is ok, our method offers greater flexibility and consistently delivers better results.

Reviewer E9fU6/10 · confidence 2/52024-07-13

Summary

The paper identifies a limitation in RLHF methods, noting that ranking over pairs of trajectory segments often fails to capture the varying strengths of preferences across different pairs. To address this, the paper proposes a new adaptive preference loss (Ada-DPO), underpinned by distributionally robust optimization (DRO). The proposed method improves policy performance and is supported by theoretical proofs on convexity and univariate analysis.

Strengths

- The paper identifies a key limitation in previous RLHF methods and provides a clear theoretical analysis of its methods and claims. - It presents robust experiments and quantitative analysis on both robotic controls and natural language generation tasks.

Weaknesses

- It seems that only a single run was conducted for the experiments in the NLP tasks. More repeated runs would be more convincing, especially since the differences in performance are so close. Additionally, p-value testing for that domain would help determine if the differences are truly significant.

Questions

- I would like to see more examples like Figure 7, comparing the learned reward differences with different scaling factors.

Rating

6

Confidence

2

Soundness

3

Presentation

3

Contribution

3

Limitations

- More discussion on the limitations of the proposed method and future directions can be added.

Reviewer RUFd5/10 · confidence 4/52024-07-15

Summary

The paper focuses on redesigning the loss function with adaptive scaling parameters to deal with the uncertainty in the preferences and thus improving the reward modeling flexibility. In the context of both robotics and NLP, the algorithm with the new loss shows improved performance.

Strengths

1. The proposed loss is flexible and can be incorporated in majority of the current RLHF frameworks 2. The objective has strong connections to DRO which is critical since preferences will be noisy and sub-optimal in practical scenarios.

Weaknesses

1. The experimental ablation is weak and doesn't provide concrete indications of the strength of the proposed algorithm. For ex in Figure 3, the win-rate gap is marginal and not significant. Why is the performance in summarization better in Figure 4 is not extremely clear? 2. The experimental evaluations lack important benchmarks and comparisons. For the Robotics environment, it's crucial to compare with Pebble, SURF, and PARL to understand the true efficacy of the proposed approach.

Questions

1. Can the authors provide more concrete mathematical justifications regarding the issue with linear depedence with the logit of the preference distribution? Specifically, connecting with the relation to noisy preferences? 2. "For some applications, it may lead to a reward function that is not flexible enough to differentiate a 141 pair of segments, which are supposed to have significantly different rewards." Can the authors please highlight such examples intuitively or mathematically, why such a collapse might happen? 3. The authors mention "Our proposed method is inspired by DRO, it serves a distinct purpose: improving reward learning in RLHF, which is orthogonal to distributional robustness". Can the authors pls justify this statement and discuss why it's orthogonal? DRO in reward learning will also improve the robustness and flexibility of the reward model to noisy preferences, in what sense it does better than DRO? 4. Can the authors provide a more concrete definition of the best win rate and best prediction accuracy? 5. It will be helpful if the authors can provide more details on how the trajectories are constructed for the robotics experiments.

Rating

5

Confidence

4

Soundness

3

Presentation

2

Contribution

2

Limitations

Check above

Reviewer 4UPH2024-08-10

Response to the authors

Dear authors, Thank you so much for the helpful responses. I hope these discussions and revisions can be incorporated into the next version of the manuscript. I've increased my rating to 6.

Authorsrebuttal2024-08-11

Thank you for the thorough review

Dear Reviewer, Thank you for your thoughtful feedback and for increasing the scores toward acceptance. We will incorporate your suggestions in our next version. Best regards, The Authors

Reviewer ZDuk2024-08-12

Response to the authors

Thank you for the informative response! Can you further visualize situations in which the BT model cannot express, maybe in your synthetic setting where the true preference strengths can be captured? The learned scaling parameter also seems linear regarding preference strength, as shown in Figure 5(b). If this motivation is not well-established, adding Adaptive Preference Scaling that has complicated implementation is not worth the cost. I also wonder if using simpler approaches, such as adding a reward margin to ranking loss or adding another layer of (non-) linear function outside the sigmoid function, can achieve similar or better performance.

Authorsrebuttal2024-08-13

Thank you for the discussion

Thank you for your thoughtful comments! Due to NeurIPS regulations, we are unable to upload figures or provide links. However, we would like to clarify the following points: The BT model assumes a linear relationship between the reward difference and the logit of the preference distribution. This means it cannot accurately represent situations where the relationship is non-linear. In contrast, our method models this relationship as non-linear. Specifically, with linear regularization, the relationship becomes piecewise-linear (see Proposition 3.1), and with quadratic regularization, the relationship is more complexly non-linear (see Proposition 3.2). In both cases, it is important to note that when the logit value is small, our method learns a smaller reward difference. Conversely, when the logit value is large, our method learns a larger reward difference compared to the BT model. This adaptive, non-linear relationship makes our method more flexible and better suited for capturing complex preference dynamics that the BT model cannot handle. Further examples of the benefits of this non-linear relationship are provided in our response to Q1 of reviewer E9fU. In Example 1 (low preference strength), Example 2 (moderate preference strength), and Example 3 (large preference strength), the reward differences for DPO are 0.64, 1.07, and 1.47, respectively. Meanwhile, the reward differences for Ada-DPO are 0.31, 0.88, and 2.83, respectively, demonstrating that Ada-DPO scales more appropriately across varying levels of preference strength. Regarding the learned scaling parameter shown in Figure 5(b), we want to clarify that the near-linearity of the scaling factor does not imply a linear relationship between the learned logits and preference strength. Lastly, in the additional experiment, we implemented the reward margin method from [1] on three robotic control tasks. We did not include other reward margin methods, such as those from [2] and [3], because they require additional data called "score", which is expensive to obtain, making a fair comparison difficult. As shown in the table below, the reward margin method from [1] not only fails to match our method’s performance but also performs significantly worse than standard RLHF. | Task | Method | Return | |--------------|----------|---------| | HalfCheetah | Pref | 2724.42 | | | Margin | 577.98 | | | Ada-Pref | **2875.45** | | Ant | Pref | 2917.81 | | | Margin | 866.5 | | | Ada-Pref | **3177.11** | | Hopper | Pref | 1324.91 | | | Margin | 40.13 | | | Ada-Pref | **1692.1** | [1] Qin, Bowen, Duanyu Feng, and Xi Yang. "Towards Understanding the Influence of Reward Margin on Preference Model Performance." arXiv preprint arXiv:2404.04932 (2024). [2] Touvron, Hugo, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov et al. "Llama 2: Open foundation and fine-tuned chat models." arXiv preprint arXiv:2307.09288 (2023). [3] Amini, Afra, Tim Vieira, and Ryan Cotterell. "Direct preference optimization with an offset." arXiv preprint arXiv:2402.10571 (2024).

Reviewer RUFd2024-08-12

Response to Rebuttal by Authors

Thanks for providing concrete answers to my queries and issues. Most of my concerns have been resolved, hence I am increasing my score. Please add the references mentioned in detail and explain the reason for not comparing them with the baselines. However, mathematical justifications regarding the issue with linear dependence with the logit of the preference distribution is not very clear and would request authors to provide a clear explaination mathematically.

Authorsrebuttal2024-08-13

Thank you for the discussion

Thank you for your valuable comments. We will include the mentioned references and the discussion in the paper. Additionally, we would like to address your remaining concern regarding the mathematical justification related to the issue of linear dependence. The BT model assumes a linear relationship between the reward difference ($\Delta r$) and the logit of the preference distribution ($\ell$), expressed as $\Delta r = \ell$. This assumption inherently limits the model’s ability to accurately capture scenarios where this relationship is non-linear. In contrast, our method models this relationship as non-linear. Specifically, in the case of linear regularization, the relationship is piecewise-linear and can be expressed as: $\Delta r = \tau_0\ell , \text{if } -t < \ell \leq t; \Delta r=\tau_\max\ell, \text{otherwise},$ where $t$ is a threshold related to $\rho$, and $\tau_0 < 1 < \tau_\max$ (see Proposition 3.1). For quadratic regularization, the relationship becomes even more complexly non-linear (see Proposition 3.2). Importantly, in both cases, when $\ell$ is small, $\Delta r$ is smaller, and when $\ell$ is large, $\Delta r$ is larger compared to the BT model. This adaptive, non-linear relationship enhances our method’s flexibility and allows it to better capture complex preference dynamics that the BT model cannot. Although we intended to include visualizations of these relationships to make them more intuitive, NeurIPS regulations prevent us from doing so in the rebuttal. However, we will add these plots in the paper to provide better clarification. Additionally, please refer to the examples on the summarization task in our response to Q1 for reviewer E9fU, which further illustrate how this non-linearity adapts to the data. Lastly, we noticed that, despite your mention of potentially increasing the score, it has remained unchanged. We would greatly appreciate it if you could reconsider the score in light of the explanations we provided.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC