Thank you for your constructive review. We are pleased to know that you found the question we are investigating to be “interesting” and our findings to be “useful”!
---
> It is unknown that how the context prompt (e.g., [s] plays the) affects the conclusion. For example, will the conclusion hold if we change some other contexts that express same meaning of relation?
During early versions of our experiments, we averaged results over multiple different prompts to account for potential difference in LRE performance. We later decided to use only one prompt per relation due to time constraints and because we noticed similar LRE performance across different prompts. We realize this confound is important to account for, so we’ve added Table 7 to the appendix which shows how faithfulness and causality scores vary when LRE is estimated with different prompt templates. Ultimately, we find that most scores do not change dramatically.
---
> In term of handling objects that have multiple tokens, I saw in Table 4: only the first token was used to determine correctness. Do we apply this strategy to all experiments? Will this lead to false positives? For causality experiment, do we also use the first token of o’ for experiment?
We *always* use the first token for the objects that get tokenized to multiple tokens. This is true for the causality experiments as well. For some specific relations this strategy can indeed lead to false positives, though we find in practice that such collisions are rare.
To clarify this, we have made two adjustments to our submission. First, we now discuss this issue explicitly in the limitations. Second, we have included a new table (Table 9) in the appendix which shows that first token collisions are rare: on average, over $93$% of the objects per relation in our dataset can be uniquely identified with their first token. It is worth noting that this number is skewed downwards by relations such as *person mother*, *person father*, and *president birth year*. And, the LRE performs poorly on these relations anyway.
---
> Do we have some examples showing that `when LRE can not fully capture the LM's computation of the relation, the linear approximation can still perform a successful edit`?
Here is one illustrative example where the LRE is not faithful yet we have observed a successful causal edit. Given the prompt `"Malcolm MacDonald works as a"`, GPT-J correctly predicts the expected answer, `" politician"`. However, an LRE (estimated based on the following examples) will assign the highest probability to `" writer"`.
```
[
(subject='Janet Gunn', object='actor'),
(subject='Tony Pua', object='politician'),
(subject='Valentino Bucchi', object='composer'),
(subject='Christopher Guest', object='comedian'),
(subject='Giovanni Battista Guarini', object='poet'),
(subject='Menachem Mendel Schneerson', object='rabbi'),
(subject='Cristina Peri Rossi', object='novelist'),
(subject='Nils Strindberg', object='photographer')
]
```
Although the correct answer `" politician"` is within the top-5 prediction of the LRE, our faithfulness metric stringently measures top-1 accuracy and will consider it a failure case.
In contrast, when we compute the edit direction as $\Delta \mathbf{s} = W_r^{-1}(\mathbf{o}' - \mathbf{o})$, we find that $\mathbf{s} + \Delta \mathbf{s}$ is often enough to change the LM output from $o$ to $o'$ even when LRE($\mathbf{s'}$) does not assign highest probability to $o’$.
Continuing this example: LRE failed to predict the correct occupation of $s’$ = `"Malcolm MacDonald"`. However, we can use GPT-Js object representation $\mathbf{o'}$ corresponding to its encoding of Malcom Macdonald’s profession to successfully change the profession of $s$ = `"Walter Hines Page"`, from $o$ = `" journalist"` to $o’$ = `" politician"`, which matches the correct profession of `"Malcom Macdonald"`, even though LRE($\mathbf{s’}$) incorrectly predicted `" writer"`.
When causality exceeds faithfulness, it reflects relations for which such examples are prevalent.
---