Thank you for your detailed comments and for recognizing the importance of debiasing in LLM research. We organize the rebuttal in three parts to comprehensively answer the raised concerns related to: 1) LM performance, 2) clarifications about concepts and equations in section 4; 3) the novelty of our approach.
## Firstly, we want to address the concern about LM performance that might be an effect of misunderstanding our ablation setting:
> Figure 3 (b) seems to indicate that to reduce the bias, the cost is to get a much worse language model.
Figure 3 (b) illustrates ablation experiments with varying intervention layer numbers. The significant drop in perplexity occurs notably for 10 or 11 layers. It's crucial to note that, as stated in the beginning of section 4.4: "We apply DAMA to MLPs in approximately one-third of the model’s upper layers (in LLaMA 7B layers 21 - 29 out of 32),” equating to 9 layers. The corresponding perplexity is denoted with a star on the figure, which is close to the score of the original model.
## Secondly, we address your questions regarding concepts in Section 4. Moreover, we refined the section for better clarity and uploaded it to open review.
> "Certain concepts [..] are either inconsistent or not explicitly defined."
We clarified in section 4 that:
- “stereotypical keys” and “stereotyped keys” are the same entity. We use just the former name.
- “gender values” and. “gendered values” are the same entity. The latter name is now used exclusively.
- “Grammatical gender” is the quality manifested by the specific inflection of words. We now use a descriptive definition of grammatical gender to avoid confusion with gender scores.
> Questions regarding Equation 2
$z$ is the latent vector, specifically the output of the lth feed-forward layer, as denoted in the subscript $FF_{out,l}=z$. We used it just as a loss argument to find the gendered value vector $v_o$
We added $\text{argmin}$ notation in Equation 2 to clarify that. The missing parenthesis was also fixed.
> "The first two items in equations look similar to variational inference."
In equation 2, we consider the model's output probability $P(o'|X')$ conditioned on a known prompt ( $X'$). The formula resembles variational inference, yet there is a crucial difference: $X'$ is a known prompt, not a random variable. As a result, a prior distribution cannot be defined here except for a trivial one.
> Meaning of metrics $U$,$V$, and $P$:
We just want to clarify that $U$ and $V$ are matrices of concatenated stereotypical key vectors and gendered value vectors as described in the article (those are not metrics). We added further explanation regarding equation 5 in the revised paper.
## Lastly, we’d like to highlight the novelty of DAMA presented in the Discussion Section.
> Novelty in the context of model editing methods
Previous model editing methods targeted highly specific information encoded in the model ( Mitchell et al., 2022; De Cao et al., 2021), while we introduced a method capable of editing general dataset artifacts, such as various manifestations of gender bias. In Table 2, we show that DAMA significantly outperforms MEMIT (Meng et al., 2022 ) in this aspect.
> Advancements in projection-based debaising.
In contrast to earlier projection-based debiasing methods that focused on learning and projecting latent representations for specific gender-related tasks (e.g., gender classification in bios Ravfogel et al. 2020), our method demonstrates that projection can be achieved without relying on auxiliary tasks. Notably, the projections in our approach are directly applied to the model component, avoiding alterations to the architecture or parameter size. Our theoretical and empirical evidence further establishes that applying debiasing projections on MLPs effectively reduces bias while preserving other encoded information, showcasing the generalizability of the method across tasks.
We hope that the response addresses most, if not all, of your concerns. We are open to further discussion, suggestions, or questions you may have.