AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback

Large language models (LLMs) such as ChatGPT have seen widespread adoption due to their strong instruction-following abilities. Developing these LLMs involves a complex yet poorly understood workflow requiring training with human feedback. Replicating and understanding this instruction-following requires tackling three major challenges: the high cost of data collection, the lack of trustworthy evaluation, and the absence of reference method implementations. We address these challenges with AlpacaFarm, a simulator that enables research and development for learning from feedback at a low cost. First, we design LLM prompts to simulate human feedback that are 50x cheaper than crowdworkers and display high agreement with humans. Second, we propose an automatic evaluation and validate it against human instructions obtained on real-world interactions. Third, we contribute reference implementations for several methods (PPO, DPO, best-of-n, expert iteration, and more) that learn from pairwise feedback. Finally, as an end-to-end validation of AlpacaFarm, we train and evaluate eleven models on 10k pairs of real human feedback and show that rankings of models trained in AlpacaFarm match rankings of models trained on human data. As a demonstration of the research possible in AlpacaFarm, we find that methods that use a reward model can substantially improve over supervised fine-tuning and that our reference PPO implementation leads to a +10% improvement in win-rate against Davinci003. We release all components of AlpacaFarm at https://github.com/tatsu-lab/alpaca_farm.

Paper

References (76)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer NPeD7/10 · confidence 4/52023-06-29

Summary

This paper presents a simulation framework, AlpacaFarm, for developing LLMs with human feedback. AlpacaFarm adopts LLMs (e.g., GPT-4) to generate feedback (i.e., the ranking of candidate responses given the query), and evaluate the performance by calculating the win-rate against the baseline. This framework can obtain synthetic feedback data cheaply compared with human annotators. The major contribution is the system design and validation of the AlpacaFarm framework. System Design: - i) the framework constructs a relatively comprehensive evaluation dataset. - ii) multiple LLMs are adopted as annotators proxy (GPT-4, ChatGPT and Davinci003) - iii) random noise is injected to mimic the variability of human annotators. Validation: Simulated evaluation (i.e., the model performance ranking ) results of learning with the synthetic feedback dataset match that obtained by training models with human feedback and evaluated by real annotators. Besides, the pairwise ranking and variability also correlate with the human results well. Finally, the paper benchmarks the reference methods with the framework of LLAMA-7B, suggesting that SFT plays a significant role and highlights the superiority of PPO.

Strengths

- Clarity: The overall presentation is good, clearly conveying the key ideas of this paper. - Significance: This framework would potentially become a useful resource for building, and evaluating the LLMs with feedback datasets when budgets are limited. - Originality: The novelty of the framework is somewhat limited as generating synthetic feedback datasets & evaluating with powerful LLMs have already been explored recently. Yet, it is still worthwhile to explore combing these dots and justify its effectiveness.

Weaknesses

- Some hyper-parameter choices are unjustified (such as the ratio in Question 1). - The effect of this framework is under-explored when the baseline model is close to the annotators LLMs. --- After Rebuttal: Thanks for the response which well-addressed my questions well. I have raised my score to Accept.

Questions

- The paper mentioned that 25\% of the simulated preferences are randomly flipped. Any explanation on the ratio? Besides, while the results are consistent with the human results, I think this randomness may not faithfully reflect the inherent variance as human variability depends on the background and geography of the annotators. - What results would be when the base LLM is close to the annotator LLMs? Will this framework still effective when we are adopting a powerful LLM itself as annotators? - What prompt is used to generate feedback and evaluate the candidate responses? Did you do some stabilization techniques such as self-consistency in [1] to stable the evaluation results? - Typo: Captions Table 4 and Table 5 of the Appendix: longer outputs.s -> longer outputs. [1] Large Language Models are not Fair Evaluators

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

N/A

Reviewer JtTg8/10 · confidence 4/52023-07-05

Summary

This paper introduces AlpacaFarm, a simulator that enables faster and cheaper research and development of fine-tuning LLMs with human feedback. The authors propose to use an LLM to simulate human feedback, which is 45x cheaper than using crowdworkers and displays high agreement with humans. They also identify an evaluation dataset which is representative of real-world instructions from interactions between humans and chatbots or other LLMs. The authors also plan to open-source reference implementations for multiple methods which are widely used in the research and applications such as PPO, Best-of-N (BoN), and Expert Iteration (EI). The authors validate their simulator and evaluation protocols by showing that rankings of models trained in AlpacaFarm match those of models trained on human data. The results indicate that leveraging a reward model can improve performance over supervised fine-tuning, which is in line with prior work.

Strengths

## Clarity The paper is well-written and clear overall. I also appreciated that the authors didn't only consider the agreement between LLMs and humans but also the variability in the answers which is an important aspect of human evaluation and feedback. ## Impact The paper addresses a timely topic of great importance, namely understanding how to best fine-tune LLMs with human feedback. Although it is more of an engineering / datasets / benchmarks paper, I believe it fills a gap in the literature and could lower the entry barrier for doing research on RLHF and related methods which are rather complex and expensive to train well. ## Novelty Although the paper does not propose a novel approach, it open-sources reliable implementations of popular approaches for fine-tuning LLMs with human feedback and proposes an automatic (fast and cheap) way of evaluating such methods. I believe these could be quite impactful and valuable for the community. I expect this can enable research and development of new methods, as well as a platform for fairly comparing them in order to make faster progress on these problems.

Weaknesses

## Generality One of the main weaknesses of this paper is the fact that only LLaMA-based models are considered. However, the LLM simulator is advertised as a general platform for researching, developing, and evaluating human feedback fine-tuning of LLMs. In particular, one open question is whether the models you use for evaluation and simulating human feedback transfer to other base models such as OPT, Pythia etc. 1) Thus, it would be useful to include experiments validating this hypothesis to demonstrate your protocol is robust to the base model used. 2) In addition, it would also be good to include different models for evaluating and even for simulating human feedback to see how much variance there is and increase the robustness of the results obtained with this platform. In particular, I expect that the results won't perfectly match human evaluation and feedback (particularly for the absolute performance), but adding different ways of simulating human feedback and evaluating LLMs can at least decrease the variance in the final results. 3) It would also be good if the authors can include human evaluations in the paper.

Questions

1. In section 3.2, you claim that the agreement between your model and human annotators is high and quote 65%, which seems rather low to me. Can you provide more details on how this number was computed and what it means in practice? How does it compare with the agreement across humans? 2. Also in section 3.2, you mention that to emulate intra-annotator variability you inject random noise flipping the simulated preference 25% of the time. How did you come up with this number, did you run experiments to find out the intra-human variability? How did you validate that the resulting models match human variability?

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

4 excellent

Limitations

The authors mention some of the assumptions and limitations of the paper but I strongly suggest having a separate section that discusses these in greater depth. Given the potential impact of this simulator and relevance to real-world applications, it is essential for readers to understand that this is just a simulator which may be inaccurate so it should only be used for research and preliminary experiments; human evaluations and potentially feedback should always be used before deployment or to make stronger claims about a model's capabilities.

Reviewer pSDY8/10 · confidence 4/52023-07-05

Summary

The authors provide a simulator for experiments with LLMs that aim to learn from human-feedback, in particular, human binary comparisons. This allows researchers to run exploratory experiments with, e.g., RLHF, quickly and cheaply, without having to collect human data. The main contribution is the open source library that includes various components needed for these type of experiments. In the paper, the authors evaluate their simulator; using end-to-end validation, validating subparts of the library, and showcasing how it can be used to produce findings match those found when using human feedback.

Strengths

Learning from human feedback is an important topic that has long been hard to study for any but the most well-resourced labs. Alpaca Farm will enable more research on this topic Open source software that enables other researchers is always great. The paper is well written. Thorough validation. The validation results, especially Figure 2, are impressive. The finding in Figure 3 (reward model over-optimisation only happens with strong inter-rater disagreement) is intriguing (seems worth further study)

Weaknesses

From more to less relevant concerns. I will raise my score if these are addressed sufficiently in the rebuttal. The paper could explore the limitations of Alpaca Farm more. I don't actually believe that Llama-7B with some some instruction fine-tuning and a bit of PPO is comparable to GPT-3.5; you probably don't believe this either. However, Table 1 shows a winrate of 55% of "PPO". This is probably because your evaluation instructions are relatively easy, only single turn, you don't factor in adversarial inputs, and so on. This is fine, but it should be discussed more in the paper. In general, the paper would be better if it elucidated the limitations of your simulator a bit more, as there clearly are some. Try to break it, show not only where it works, but where it stops working. There has been recent work that shows that MTurk crowdworkers use AI a lot. https://arxiv.org/abs/2306.07899 . So, it's not wonder that MTurk evaluations agree with the evaluations from you GPT evaluator models. How do you expect this to affect your results? Figure 4: this experiment would be better if you used humans to evaluate the winrate on the demo. The x-axis is the evaluation in Alpaca Farm. The y-axis should be the "realistic evaluation", which means realistic data AND human evaluation. Why does Figure 2 have 10 data points? What are these data points? Comparing to Table 1, it seems like you included GPT-4 on Figure 2; this seems misleading, as this only measures the evaluation part of your simulator, and not the training part. You should show a Figure where you only include models that were actually trained on the simulated preferences. I have some worries thinking about that the training signal (pairwise comparisons) and the evaluators (pairwise comparisons) come from the same API-models. This might lead to overfitting to these particular evaluators. However, in practicse, this doesn't seem to be an issue (see Figure 2). Of course, if you work with humans, you may also use the same humans for training signal and eval (although best practice would probably be to use different humans) Paper could be shorter, a bit much repetition. ------- EDIT: some of these points have been convincingly addressed in the rebuttal, so I raise my score

Questions

Why does Figure 2 have 10 data points? What are these data points? Comparing to Table 1, it seems like you included GPT-4 on Figure 2; this seems misleading, as this only measures the evaluation part of your simulator, and not the training part. You should show a Figure where you only include models that were actually trained on the simulated preferences. There has been recent work that shows that MTurk crowdworkers use AI a lot. https://arxiv.org/abs/2306.07899 . So, it's not wonder that MTurk evaluations agree with the evaluations from you GPT evaluator models. How do you expect this to affect your results?

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

The paper could explore the limitations of Alpaca Farm more. I don't actually believe that Llama-7B with some some instruction fine-tuning and a bit of PPO is comparable to GPT-3.5; you probably don't believe this either. However, Table 1 shows a winrate of 55% of "PPO". This is probably because your evaluation instructions are relatively easy, only single turn, you don't factor in adversarial inputs, and so on. This is fine, but it should be discussed more in the paper. In general, the paper would be better if it elucidated the limitations of your simulator a bit more, as there clearly are some. Try to break it, show not only where it works, but where it stops working.

Reviewer goj18/10 · confidence 4/52023-07-07

Summary

The paper identifies three major challenges in training models with human feedback: (a) the cost of *preference* data collection, (b) the lack of trustworthy eval, and © the absence of implementations for reference methods. I completely agree with the fact that the process of training LLMs with human feedback is less understood due to the lack of published information and tools on them. The paper does a great job at addressing these dimensions in great detail with solid experimental results and thought-provoking findings. The paper establishes AlpacaFarm as a framework whose feedback and evaluation is synergetic with training on human feedback and human evaluation. Overall, the paper is well-written and clear to follow. *Cost of preference data collection* - The paper’s contribution in creating prompts for API LLMs that follow high human agreement and replicate human variability is very novel! - I do not fully understand why the authors want the practitioners to collect high quality human feedback (post-AlpacaFarm simulations) if they already have AlpacaFarm API LLM annotators. It makes it difficult for me to digest how Alpaca Farm is reducing the cost of collecting preference data if eventually we do need to collect actual human feedback. - I will be interested in understanding the gap between the performance of the methods trained with API LLM feedback vs human feedback by fixing the evaluator as Alpacafarm eval in the first case and humans in the other. Currently, the presented results focus on the gap between training with the simulated feedback + simulated evaluation and human feedback + human evaluation. *Trustworthy eval* - The experiments establish the high correlation between the simulated win-rates and human win-rates. It was interesting to observe that the rankings of the methods match well under the simulated environment and the real-world environment. - I feel the paper lacks a discussion on evaluating the LLMs on the existing NLP datasets such as MMLU, SuperGLUE and benchmarks such as BigBench. I understand that 805 instructions may be a set of instructions that humans care about but I still feel that the prior works on creating datasets for model eval deserve credit in the main text. *Reference Implementations* - I agree that there are not a lot of implementations for reference methods, and really appreciate the authors for providing them. - Given, PPO is known to be finicky and hard to stabilize, it would have been good to get some more details about the hyperparameter search in the main/supplementary material. - Minor comment: It would be good to mention that Best-of-n policy uses n = 1024 more frequently in the plots.

Strengths

Mentioned in the main comment.

Weaknesses

No major issues with the contribution and experiments.

Questions

More details on the reference method implementations in the main/appendix would be great.

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

No major issues.

Reviewer pSDY2023-08-10

Updated Figure

> We agree and have updated the figure in the main paper Could I see the updated Figure please?

Authorsrebuttal2023-08-10

Link to figure & table

Please refer to this [anonymized link]( https://anonymfile.com/g3xjN/alpacafarm-graypoints.pdf) for a PDF containing the updated figure and table from the main manuscript. We apologize for the oversight in not providing any proof of our claim.

Reviewer pSDY2023-08-10

Response to figure

OK, this still looks good. Overall, you might benefit from experiments that train and evaluate a model with poor performance, so that the left side of the graph also has a blue dot. In total, the response has updated me enough to raise my score. It seems that I can't edit the original review right now, but I assume this will become available later.

Authorsrebuttal2023-08-13

Thank you for considering the rebuttal and for your quick answer!

Reviewer JtTg2023-08-11

Response to Rebuttal

I thank the authors for their thorough responses and for going the extra mile to further improve the paper by running additional experiments and including an extensive discussion of its limitations. I also appreciate the emphasis of some details and results I might have missed in the Appendix, for using a wide range of models to perform evaluations, and for releasing human annotations. I'm happy to say the rebuttal has addressed my main concerns. I believe this paper would be a very valuable contribution to the community by democratizing research on an important topic (RLHF) which has a relatively high entry barrier. I also find the experiments to be very thorough. In light of this, I have increased my score to 8 to reflect my strong support of this paper.

Authorsrebuttal2023-08-13

Thank you for considering our rebuttal and for the encouraging words!

Reviewer goj12023-08-12

Response to Rebuttal

Hi, I thank the authors for rebuttal. - It is interesting to see that the optimal LLM feedback setups are different if you want to use them for supervision versus simulators. On "affordable method development with AlpacaFarm as a simulator": - I believe that the AlpacaFarm's "cost-effectiveness" would lie in their faithful automatic evaluation that achieves high agreement with the human judgments, instead of AlpacaFarm's feedback data itself. - Let us say, a practitioner has N plausible RL algorithms for aligning their LLM. Without any doubt, collecting feedback data (pairwise judgments) from AlpacaFarm will be cheaper than human feedback. - **Scenario 1**: Under the cost-effective argument in the paper, the practitioner would test their RL algorithms on AlpacaFarm feedback data, and use the best method (on AlpacaFarm eval) to train on human feedback data. Finally, they would perform a human eval of the algorithm trained with human feedback. - **Scenario 2**: The practitioners could collect human feedback data and select the algorithm that works best on AlpacaFarm eval. Finally, they can perform a human eval of the best algorithm thus found. - In my opinion, **Scenario 2** is more straightforward and cheaper than **Scenario 1**. Overall, I am satisfied with the paper's findings and experiments. Good luck to the authors. I will keep my scores unchanged since there were not any major concerns in the original rebuttal anyway.

Authorsrebuttal2023-08-13

Thank you for the detailed answer. This is a great point! We agree that Scenario 2 is cheaper in the case of a single-round RLHF, which is the experimental setting we consider! In the case of multiple RLHF rounds, scenario 2 would likely become more expensive as a different set of human preferences would need to be collected for every considered model! We hope that future work will consider AlpacaFarm in multi-round settings.

Area Chair 7zSo2023-08-17

Questions from area chair

Hi authors, To facilitate discussion about the paper, it would be great if you could answer the following question. You use a constant label flip rate of 25%. This seems to ignore the fact that different prompts x will have different levels of human ranker variability. For example preferences about prompts that relate to taste ("Is rock better than jazz etc.") will be highly variable, whereas math problems ("does 2+2 =4?") will have zero variability in responses. Can you comment on this? I am not saying this is a huge problem, but wanted to have your opinion.

Authorsrebuttal2023-08-18

Thank you for your insightful question. We recognize and agree that the degree of randomness depends on the example. Although the constant label flip rate of 25% doesn't directly address this variability, we have incorporated two mechanisms to simulate such example-specific randomness: 1. **Sampling the annotator**: For each annotation, we sample a different simulated annotator. This strategy is designed to capture the variability between annotators, especially for more subjective queries like "Is rock better than jazz?". 2. **Sampling from each annotator**: In some examples (e.g. Question: "Is 2+2=4?", Answer 1: "Yes", Answer 2: "yes") the choice of the preferred answer possess inherent randomness even for a given annotator. In such instances, our simulated annotators have two sources randomness. Firstly, we sample responses from all annotators with a temperature of 1.0. Secondly, we randomize the order in which answers appear in the prompt. Both of these methods should not influence the annotator's preference if that preference is clear. However, in scenarios where preferences are ambiguous, they lead to high randomness. The constant label flip rate of 25% acts as an example-agnostic randomness source. While it might seem excessive for questions with objective answers (e.g., Q:"does 2+2 =4?", A1: "Yes", A2: "No"), our experimental observations suggest this rate is necessary to emulate human variability (refer to our [general rebuttal](https://openreview.net/forum?id=4hturzLcKX&noteId=Hq0kpzSHSw)). Qualitatively, we have also found human annotations to exhibit high variability even when there was an objectively better answer. This might be attributed to the typically lengthy responses which annotators might not scrutinize rigorously, thereby overlooking occasional errors. We acknowledge that our method of emulating human variability has many limitations, which we discuss in the new limitation section. We hope that it will serve as a preliminary step in sparking the desired discourse and are eager to see how the community further refines this approach!

Program Chairsdecision2023-09-21

Decision

Accept (spotlight)

© 2026 NYSGPT2525 LLC