Response
Thanks for your helpful comments! Below we address your concerns.
### Q1: Limited evaluation
Thanks for the comments. We have added more evaluation results on the Open LLM Leaderboard benchmark [1] that consists of four tasks, ARC, HellaSwag, MMLU, and TruthfulQA. Additionally, we have enriched our evaluations by training more pre-trained LLMs on our selected data, such as LLaMA-2-13B and Mistral-7B besides LLaMA-1-13B in the original submission. We have summarized these results in the table of the general response, while the detailed results are in Table 3,4,5 of the paper. On the Open LLM Leaderboard, our approach outperforms the random baseline and other strong models across three different backbone models. However, we would like to note that the performance on benchmarks such as MMLU is largely determined by the underlying base model, thus the gains after alignment tuning is typically not large, unless the training data is specifically crafted for the benchmark. This is why recent works on alignment put more emphasis on MT-bench and AlpacaEval [2].
Notably, on MT-bench and AlpacaEval, our added results show that our method with the LLaMA-2-13B model surpasses Vicuna-13B-1.5 that trains on 10x more data, and LLaMA-2-13B Chat that has undergone extensive RLHF training. With Mistral-7B, our approach attains an MT-bench score of 7.29 with only 10k training data, marking a state-of-the-art performance on MT-bench *for all open-source SFT models at 7B and 13B sizes*. We also observe that, while the gains of the Mistral DEITA model on AlpacaEval are slightly lower than those on MT-bench, a deeper analysis (as depicted in Figure 2) indicates that the Mistral DEITA model excels particularly in mathematical and complex reasoning tasks, areas not primarily covered by AlpacaEval.
### Q2: Marginal performance improvement. DEITA is worse than Vicuna-13B on AlpacaEval dataset. With human evaluation, Vicuna performs almost similar to DEITA.
We emphasize that the purpose of this work is not to set up a new record on the performance, but to achieve top results with much less data, for example, the training of DEITA is far more efficient than Vicuna by using 10x less training data. Furthermore, we note that Vicuna is a SOTA SFT model, and DEITA outperforms it on most of the cases across three different benchmarks and various backbone models (see the table in General Response), except for just one setting on AlpacaEval – **DEITA is the first automatic data selection approach that achieves comparable performance to SOTA models such as Vicuna and WizardLM with 10x less data**. Therefore, we view comparable performance to Vicuna as the strength of our approach, rather than weaknesses.
### Q3: Missing comparison with instruction selection works
Thanks for the suggestion! In the original submission, we have shown that “direct scoring” from Alpagasus [3] does not work well in Table 2 and 3, and using ChatGPT to score all the samples in the data pool following their paper is too expensive, thus we did not explicitly include them in the main tables. Following the reviewer’s advice, we have added the Alpagasus and LIMA baselines, the comparison of different instruction selection approaches is shown below (based on LLaMA-1-13B):
| **Model** | **Data Size** | **MT-Bench** | **AlpacaEval(%)** |
|----------------------------|---------------|--------------|-------------------|
| Random | 6K | 5.84 | 73.91 |
| Alpagasus | 6K | 5.61 | 71.21 |
| LIMA | 1K | 4.29 | 41.98 |
| $\text{TagLM}$ | 6K | 6.09 | 72.80 |
| $\text{DEITA}_{6K}$ | 6K | **6.46** | **77.08** |
We see that DEITA outperforms other instruction selection baselines significantly. We have included the table above as Table 5 in the paper.
### Q4: Why the complexity measurement is obtained from “instruction”, and the quality measurement is obtained from “response”?
We measure complexity from instruction only since the evolution procedure of complexity directly follows WizardLM, which takes only the instruction as input and evolves it step by step to increase the instruction complexity. For quality, our intuition is to measure how effectively a response addresses the query, thus both the instruction and the response are inputs to the scorer. We recognize that in Figure 1, the instruction input was inadvertently omitted in the depiction of the quality scorer. This was our oversight, we have slightly revised Figure 1 to be more clear on this.
[1] Beeching et al. Open LLM Leaderboard. https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard. 2023.
[2] Tunstall et al. Zephyr: Direct Distillation of LM Alignment. https://arxiv.org/abs/2310.16944. 2023