Scalable Model Editing via Customized Expert Networks

Summary

This paper presents an LLM editing method using customized expert networks and indexing neurons. Experiments have been conducted to demonstrate the effecitveness of the proposed method.

Rating

6

Confidence

4

Ethics flag

1

Reasons to accept

The idea of using customized expert networks for model editing seems novel to me.

Reasons to reject

The evaluation part of this paper can be further improved. 1) Some ablation studies are expected to analyze the effectiveness of different component of the proposed method, such as the different losses in Eq.(8). 2) How about the space and time efficiency of the proposed method since a separate expert network is necessary for each edited knowledge?

Summary

The paper proposes a method for continual model editing called SCEN. Like GRACE, it only modifies the l-th layer (a hyperparameter), specifically the down projection matrix, for each edit t which is then stored in a database (an "expert"); after T edits, at test time, the model must decide which expert to use (or none). GRACE does this by codebook maintenance, which requires no further training. SCEN proposes to do this by training an expert selector. There are presumably many ways to parameterize the selector, but the paper loosely follows the training scheme in T-Patcher to train a scalar projection for each edit to activate for that edit (by contrastive loss), then at test time uses the argmax of the projection for the given query if the value is above some threshold. Experiments are conducted on ZsRE (QA) and GRACE's version of hallucination task (reducing perplexity on edits while maintaining it on non-edits), using Llama-2. SCEN performs better than GRACE on edit success and generalization, but worse on locality; on average SCEN performs better.

Rating

5

Confidence

4

Ethics flag

1

Reasons to accept

- A reasonable modification of GRACE (vector codebook -> down projections, Euclidean distance -> trained expert selector). The method is also simple. - Better performing than GRACE in the average over reliability/generalization/locality.

Reasons to reject

- Somewhat limited novelty, which is a general problem with many variations of model editing methods (slight differences in parameterization/activation approach, which are often almost certainly over-optimized for small-scale models and datasets). - No comparison with T-Patcher, which doesn't make sense to me. It has the same continual model editing setup, and this paper is inspired by it. - More about T-Patcher: it bakes in the activation for the right expert into training. So even though it doesn't explicitly predict the expert at inference time (i.e., the neurons are not stored in a database but part of the model), it's kind of doing the same thing? T-Patcher is supposed to be also scalable (i.e., the added neurons are a minor portion of the model for thousands of edits). - Limited tasks: why only ZsRE and Hallucination, when the main baseline GRACE also includes SCOTUS? - Some writing issues (unnatural)

Summary

The paper presents SCEN, a two-stage model editing approach for large language models that improves upon outdated knowledge and hallucinations. SCEN stands out for its custom expert networks per knowledge piece and indexing neurons that control expert activation, leading to state-of-the-art results in question-answering and hallucination reduction datasets.

Rating

6

Confidence

3

Ethics flag

1

Reasons to accept

* The paper introduces a novel method for model editing that significantly improves upon existing techniques. * The two-stage continuous training paradigm is promising, combining the benefits of specialized expert networks with a dynamic neuron indexing mechanism to apply edits selectively.

Reasons to reject

* While you have mentioned T-Patcher, it is essential to provide a comprehensive overview of these related works and highlight the differences between your paper and theirs. * Consider including an experimental discussion on the effects of fine-tuning the attention layer or the $W_{up}$​ weight. This exploration will provide valuable insights into the performance improvements achieved by these modifications. * In Tables 1 and 2, the model performance based on Llama2-13B as the baseline is generally much worse than that based on Llama2-7B. The authors could explain or reasonably speculate on the reasons behind this.

Questions to authors

Please refer to the section titled 'Reasons for Reject' for further details.

Summary

This paper proposes a two-stage editing method based on expert networks, called SCEN, in the “sequential” setting that a sequence of edit requests is presented. Overall, the proposed two-stage method, the idea of incorporating the FNN value parameters as expert networks with the indexing neurons, is interesting and novel, showing the effectiveness in the experiment settings. A work is a reasonable extension of T-patcher, towards maintaining multiple knowledge neurons and experts. However, the papers are not very convincing on methods and experiment settings as follows: it is not convincing why two-stage methods are required, so comparing to T-patcher, architectural advantages are not clear. Another problem is that the proposed method seems not to be scalable well, as the number of additional indexing and weight parameters is just increasing monotonically for larger scale of edits. A simple extension of T-patcher which is not in two-stage manner needs to be compared. Additionally, the paper needs to be rewritten, particularly because the method of the first stage is not very clear (how to train FNN’s down projection weight). Nevertheless, given its positive aspects of the paper, I am currently leaning to the acceptance, while the current decision is likely changed to lower scores.

Rating

7

Confidence

4

Ethics flag

1

Reasons to accept

1. The proposed two-stage editing incorporating with the expert networks is interesting and novel, while the value parameters of FNN are considered as parameters of each expert. 2. Experiment results show that the proposed methods are promising, comparing to other various prior methods, using the llama2 models.

Reasons to reject

1. For each edit request, the proposed editing methods monotonically create a new indexing neuron with a weight vector, and need to keep its own FNN’s value parameters. As a result, if N number of sequential edits is provided, we need to keep N indexing neurons associated with their weight vectors as well as N FNN values parameters separately. It is largely questionable whether increasing the number of parameters is scalable in the practical situation. Even edits where their knowledge and topics are very similar will have their own separate indexing neurons and FNN value parameters, causing the method inefficient. 2. In the two-stage method, the actual editing process is performed at the first stage, while an indexing neuron in the second stage just plays of an “indexing role” which turns on or off, depending on the input query. It is not clear why these two stages are necessary. Why the methods are not designed in a unified manner (without two-stages), like T-patcher? 3. In the experiments, a simple extension of T-patcher not in the “two”-stage manner needs to be compared. 4. An important part is how to actually adjust the parameters of an expert, at the first-stage. But, in Section 3.2, this part is not very clear; how to train W_down^t (down projection weight)? which layers have their own expert parameters?

Questions to authors

Please see weaknesses. Below are further comments. 1) Why counterfact dataset is not used, while it is used in most existing methods? 2) The method is expanded to other multi-hop reasoning setting, like effectively working on Mquake dataset? 3) In experiment results, only cases of 200 and 1000 edits are evaluated. How long an editing sequence where the proposed method is scalable well? Assuming that 1M or 100M edits are provided, what are the memory and inference costs in the proposed methods? In such large-scale edit, even applying Eq. (12) that selects the best expert requires non-trivial inference time.

Request for response

Dear reviewer 4Ci8, We have posted extra experiment results (e.g., a supplemented baseline and other ablation studies) and clarifications to answer your questions. We wonder if you can let us know whether our responses address your concerns. Look forward to your reply. Best regards, Authors

Request for response

Dear reviewer 5GUJ, We have posted extra experiment results (e.g., a supplemented baseline for T-Patcher) and clarifications to answer your questions. We wonder if you can let us know whether our responses address your concerns. Look forward to your reply. Best regards, Authors

Request for response

Dear reviewer amby, We have posted extra experiment results (e.g., a supplemented baseline and other ablation studies) and clarifications to answer your questions. We wonder if you can let us know whether our responses address your concerns. Look forward to your reply. Best regards, Authors

Request for response

Dear reviewer 67Nb, We have posted extra experiment results (e.g., a supplemented baseline and other ablation studies) and clarifications to answer your questions. We wonder if you can let us know whether our responses address your concerns. Look forward to your reply. Best regards, Authors

Overall Comment

We sincerely appreciate all reviewers' time and efforts in reviewing our paper. We are glad to find that reviewers generally recognize our strengths: - The proposed two-stage editing method is **interesting and novel** [67Nb,4Ci8], the proposed expert network and index neurons for implementing selective editing methods are **promising** [amby]. - The method is **simple and a reasonable modification** of the previously proposed model editing methods [5GUJ]. - Experiments on different scales of Llama2 outperform existing model editing methods [67Nb,amby], Achieved the **best balance** among the three indicators of reliability/generalization/locality [5GUJ]. We also greatly appreciate the reviewers pointing out the shortcomings of the paper, for which we have supplemented detailed experiments and discussions. The following content is a summary of our rebuttal: **(1). Detailed comparison with T-patcher** - We made comparisons on Llama2-7B and 13B, and outperformed the T-patcher method on both the 7B and 13B models. | | Reliability | Generality | Locality | Avg | |---------------|:-----------:|:----------:|:--------:|:----:| | T-Patcher-7B | 86.0 | 76.3 | 20.5 | 60.9 | | SCEN-7B | 100 | 90.0 | 83.3 | 91.1 | | T-Patcher-13B | 84.5 | 71.2 | 8.5 | 62.9 | | SCEN-13B | 99.5 | 80.7 | 83.5 | 87.9 | - We also detailed the statistics of erroneous neuron activations when continuously editing 200 times with T-Patcher and SCEN. It can be seen that SCEN has fewer erroneously activated neurons compared to T-patcher. | | False Activation(%) | Correct Response(%) | |-----------|---------------------|---------------------| | T-patcher | 71 | 48.5 | | SCEN | 7.5 | 93.5 | - We compared the GPU usage and inference time performance of T-patch and SCEN. SCEN uses less GPU memory during inference, but because it needs to schedule the expert network into GPU memory during inference, this will generate extra time consumption, so it takes longer than T-patcher. | | Ex-GPU(M) | In Edits(s) | Out Edits(s) | |-----------|-----------|-------------|--------------| | T-patcher | 121.664 | 127.7 | 126.76 | | SCEN | 20.996 | 303.5 | 125.32 | **(2). Ablation Studies** - In the paper, we use $W_{up}$ as the expert network, and we also compare the situations where other weights are used as the expert network. It can be seen that the effect is best when $W_{down}$ is used as the expert network. | | Reliability | Generality | Locality | Avg | |------------|-------------|------------|----------|------| | $𝑊_{attn}$ | 90.0 | 77.3 | 92.0 | 86.4 | | $𝑊_{up}$ | 96.5 | 83.2 | 86.7 | 88.8 | | $𝑊_{down}$ | 100 | 90.0 | 83.3 | 91.1 | - For the index neurons, we demonstrated the impact of different loss composition methods on the final effect. | | Reliability | Generality | Locality | Avg | |------------------------------------|-------------|------------|----------|------| | w/o $l_{disactivate}$ & $l_{margin}$ | 100 | 93.8 | 73.7 | 89.2 | | w/o $l_{margin}$ | 100 | 90.6 | 82.1 | 90.9 | | SCEN loss (Eq. 8) | 100 | 90.0 | 83.3 | 91.1 | **(3) Issues in the use of the dataset** Our approach primarily focuses on sequence editing, with a specific emphasis on comparing with previous sequence editing methods. Therefore, in terms of dataset selection, we follow the datasets used in previous works. Please contact us if we can do something else to help you better understand and recommend our paper.