Summary
This paper theoretically and emperically investigates the reasons why task arithmetic (an emerging technique for editing pre-trained neural networks) works.
The paper shows that, contrary to previous hypotheses [39,79,80], lienarity of the fine-tuning on individual tasks is not sufficient to fully explain the success of task arithmetic (Sec 3).
Instead, the authors propose the related and straight-forward idea of weight disentaglement (Eq 4) as an explanation (Sec 4), also providing a measure of the disentaglment error for two tasks (Eq 6).
This idea is leveraged to improve task arithmetic performance by constraining the fine-tuning to linearized models - which importantly is shown to be an improvement over post-hoc linearlization, with marginal additional computational complexity (Sec 5).
Finally, an additional connection between task arithmetic and the eigen space of the Neural Tangent Kernel is used to argue that weight disentanglement (and hence the capacity for task artihmetic) are a learned properties during pre-training, not inherent properties of parameterization or architecture (Sec 6).
The empirical results focus on the CLIP image-text Vision Transformer model family, and many experimental details are reported, inclduing in the additional material in the appendix.
Strengths
This paper is the best kind of NeurIPS paper; Beautifully written and a delight to read, the authors have considered a timely and pertinent problem in an emerging machine learning domain, applied a methodological theoretical investigation; clarified prior hypotheses in the literature; used the theoretical findings to propose a simple novel methodology, and compellingly evidenced the subsequent effectiveness with an appropriate evaluation protocol. As a cherry on the cake, the methodology is a 'drop in' method that can be applied easily to existing approaches, and actual code is provided without breaking review anonymity in the supplementary material. Bravo.
### Originality
* There are multiple practical take-aways that are immediately useful; the task disentanglement error metric (Eq 6), intuition for why task vector coefficients should be << 1 (Ln 181), simple drop-in code in the appendix that enables re-production and immediate application of the ideas (Listing 1 in the appendix).
### Quality
* The work appears to be of high quality, the metrics and emprical appear to support the theoretical findings and claims.
### Clarity
* This paper is superbly well written, and the logical argument and flow is well constructed. This paper was a delight to read and think about.
### Signifigance
* Multiple compelling and genuine avenues for future research are identfidied (Ln 239, Ln 286, Ln 315, Ln354)
* Provides timely and much-needed (i) overview of the emerging and rapidly evolving technique of task arithmetic with pre-trained models, and (ii) the beginnings of a theoretical grounding and understanding for why this technique is possible.
Weaknesses
* The main weakness of the paper in the present form is that the empirical results are limited to ViT models (specifically, CLIP). Within this modality, the empirical results are compelling (e.g. using multiple model families and 8x distinct datasets/tasks), and provide evidence for the theoretical findings, however the paper's authority and strength would be enhanced by demonstrating some of the key findings hold with another architecture or data type. E.g. I would love to see the results in Table 3 (Sec 6.2) replicated, even in a small way, with GPT or BERT on a text dataset. This would provider further strength for the empirical claims around Eigenfunction localization.
Questions
1. Ln 336 on 'Feature disentanglement'; to what degree is feature disentanglement enabled by over-parameterization / very large numbers of parameters? If this is the case, what is the parameter count relative to - i.e. some measure of the complexity of the input space or the function space represented by the data? Is there a way to quantify with a metric the 'capacity' for feature disentaglement of a given NN archiecture? These could be useful areas for investigation.
2. Related to the above point, Ln 233 notes that the advantage of the linearized model task arithmetic approach diminishes as the number of model parameters increases. I find this curious and would have expecte the opposite result - do you have any intuition why this is the case?
3. Pseudo-code or actual PyTorch code for Eq 6 would be a very helpful (and I assume straight-forward) addition to your supplementary material - this would be of immediate practical help to other researchers working on practical task arithmetic applications.
4. Related to the above, Ln 297 - can the degree of local linear independence of the NTK eigenfunctions be measured/computed with a simple metric to test this condition in other models / datasets? This would likewise be a useful contribution of the paper.
Rating
9: Very Strong Accept: Technically flawless paper with groundbreaking impact on at least one area of AI/ML and excellent impact on multiple areas of AI/ML, with flawless evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Limitations
* It's not clear to me if there are situations where the linerarization technique won't be readily applicable. E.g. are some activation functions or more esoteric neural network architectures (e.g. recurrent NN's; long-term memory components) going to problematic to linearize? Some negative examples of where NTK linearization can't be used and/or approximations or work-arounds could be a useful addition in this regard.