Response to Reviewer 6YHH
**Q1: I am concerned that the experimental design wasn't as rigorous and comprehensive as it should be.**
**A**:
The other three reviewers have all recognized the experiments conducted in our paper.
- "*comprehensive experiments*", "*interesting observations*" (Reviewer AcqK)
- "*systematic work*", "*full-stack study from prompt construction, dataset, evaluation metrics, to the quantitative and qualitative analysis*" (Reviewer 9vM7)
- "*conduct pretty solid experiments on various models and also conduct human annotation to further evaluate the results*." (Reviewer y9do).
We acknowledge that as the first paper to study social bias in code generation models, our work is not perfect in all aspects and there is still room for improvement. However, we think this could be addressed in future work.
**Q2.1: I'd like clarification on whether the RLHF effect on language model training was considered. The OpenAI help documentation for CodeX mentions it's built upon GPT-3. It's crucial to ascertain if RLHF was utilized.**
**A**: As written in our paper (footnote in Line 185), we use the latest API (code-davinci-002) of Codex to run our experiments. We suggest the reviewer check the code-davinci-002 API, which is GPT-3.5, not GPT-3. Since including links is not allowed during the rebuttal phase, we also suggest that the reviewer search for the following blog: "How does GPT Obtain its Ability? Tracing Emergent Abilities of Language Models to their Sources". This blog also mentions that code-davinci-002 is GPT-3.5.
Regarding the effect of RLHF, we think that the reviewer has some misunderstandings about our paper. The primary contribution of our paper is to **uncover and quantify social biases in code generation models**, rather than analyzing various factors such as RLHF's impact on social bias. We acknowledge the importance of the research direction highlighted by the reviewer but believe that it is not within the research scope of this paper.
**Q2.2: The authors' recent submission showcasing results for GPT-4 seems to suggest that RLHF is useful.**
**A**: Compared to GPT-3.5, there are many improvements in GPT-4, e.g., an improvement in the quality and quantity of training data, an increase in RLHF rounds, a change in alignment objectives, the use of multimodal data, and the use of MOE, etc. It's difficult to determine which factor amplifies bias and which one mitigates it. Hence, it is premature to say "RLHF is useful" without further investigation.
**Q2.3: These results appear to challenge the authors' earlier observation that larger pre-trained code generation models with more parameters tend to learn more social biases.**
**A**: We would like to clarify that this is NOT in conflict with our observations in Table 4. By saying "larger pre-trained code generation models with more parameters tend to learn more social biases in spite of better performance", we intend to convey that, **with the same model type but different sizes** (they share the same underlying architecture but with different numbers of parameters), the larger one might learn more social biases, rather than making a comparison across different types of models. For example, as shown in our experiments, CBS(**CodeGen** 6.1B) > CBS(**CodeGen** 2.7B), but CBS(**CodeGen** 2.7B) > CBS(**InCoder** 6.7B), where the larger CodeGen is more biased than the smaller CodeGen, but the smaller CodeGen shows more social biases than the larger InCoder. Different types of models (such as CodeGen and InCoder) often utilize different training data and strategies, making direct comparisons difficult. We will make the statement clearer in the revised paper to avoid misunderstanding.
Thank you for your review. Your questions and feedback have provided us with an opportunity to improve our paper and explore further in this research direction. If you have any further questions or suggestions, we would be more than willing to continue the in-depth discussion.