VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks

Summary

This paper proposes an approach to decrease the number of parameters in LoRA by introducing a collection (aka "vector bank") of sub-vectors and composing Lora matrices across modules/layers using this collection. Each sub-vector within a given LoRA module/layer is then formed as a linear interpolation of sub-vectors from the vector bank, where the coefficients are obtained by selecting top-k "basis vectors" and then applying a softmax. When applied to Roberta-large, the approach yields improvements over LoRA on the GLUE benchmark with only a tenth of the free parameters. When applied GPT-2 large , it obtains similar or slightly better performance than LoRA with around a sixth of the parameters, on the E2E task. The paper reports ablation studies showing the effect of the vector selection strategy (i.e. alternatives to top-k) and the sub-vector length.

Strengths

* Proposes a novel approach to reduce the parameter count in LoRA by sharing parameters across modules and layers using a sub-vector approach. (Originality) * The use of sub-vector style decomposition is novel in the context of LoRA and could be applied in other PEFT settings (Significance). * Demonstrates that the approach can yield improvements over LoRA using Roberta or GPT-2 with far fewer parameters (Quality)

Weaknesses

* The paper does not discuss if the proposed factorization approach (i.e. using subvectors) can be performed efficiently on accelerators such as GPUs. Also it would be good to know what is the overhead resulting from this subvector decomposition. * It would be useful to report the performance of an alternative PEFT scheme such as prefix tuning (or prefix tuning) or adapters which have a similar number of parameters as the proposed approach.

Questions

* It would be useful to discuss this paper in related work: https://openreview.net/pdf?id=qKQu1ZcJjD : The idea is to initialize matrices in an LLM using pre-trained submatrix components, which bears resemblance to the subvector decomposition. Typos: L101: * parameters -> parameter L192 * numebr -> number L314 * frozen -> frozen

Rating

8

Confidence

5

Soundness

4

Presentation

3

Contribution

4

Limitations

Yes

Summary

The authors propose an extremely parameter-efficient methods, VB-LoRA, for finetuning an LLM. Specifically, VB-LoRA has a shared vector bank (similar to a codebook), the adapter parameters (A and B) of the linear layers in an LLM are constructed from this bank by selecting the most effective bank vectors. Due to the shared mechanism, the number of trainable parameters are very little, much fewer than the strong baselines, like LoRA, VeRA and Tied-LoRA. Tha authors also validate VB-LoRA on three benchmarks (GLUE, E2E benchmark and MT-Bench) on 6 LLMs from three model families (RoBERTa, GPT-2 and Llama2). The results show that VB-LoRA's results are better or comparable to above-mentioned baselines while requiring much fewer number of trainable parameters.

Strengths

1. The proposed method, VB-LoRA, is effective, intuitive and easily applied. 2. The number of trainable parameters are very limited, mostly less than 0.1M.

Weaknesses

1. The paper is not easy to follow, especially Section 3 for the proposed method. VB-LoRA is basically an algorithm inspired by Mixture-of Expert or codebook, and is used for PEFT here. However, the authors seem to intend to make it complexer by relating VB-LoRA to other theoretical works (L137-143, L157-162), which would be better by putting in the related works. Such writing style easily distracts the reader's attention. 2. Some experimental setting is weird, not in line with previous work. - In Table 1, two high-resource tasks of GLUE, i.e. MNLI and QQP, are deleted. The authors claim that the experiments are conducted on a server equipped with 8 A100 GPUs (L215), which means the computation resource should not be a problem. Why are these two tasks are discarded? - All ablation studies are conducted on a very small task, CoLA, whose results have large variance. Normally, 24GB GPU memory and less than 1 hour are enough for finetuning RoBERTa on CoLA. Due to the small number of samples in CoLA, its results have relatively larger variance. Since the authors have much more resource, why choosing this task for ablation study? 3. VB-LoRA has a potential drawback that is inherent in its design, i.e. unscalibility. One big problem for MoE is that not all expert embeddings can be well trained. We normally include a balance loss and use a small number of experts to avoid this problem. For VB-LoRA, could you offer this ablation study? I.e. increasing the number and length of the bank vectors. 4. VB-LoRA doesn't consistently outperform baselines, or the gap is very narrow. I would be interested in more benckmarks on larger LLMs, like commonsen/arithmetic reseasoning tasks.

Questions

1. In algorithm 1, if you implement delta_W in this way, it will comsume more GPU memory during finetuning than LoRA. For implementing LoRA, the forward pass is z = Ax, H = Bz. It means we don't calculate delta_W explicitely. If you calculate delta_W, it will be stored in the memory for gradient computation, which enlarges the memoy for activation. 2. Check other questions in Weakness.

Rating

5

Confidence

5

Soundness

2

Presentation

2

Contribution

2

Limitations

The limitation discussion is not thorough.

Summary

This paper explores parameter-efficient fine-tuning (PEFT) methods in the context of further reducing the number of fine-tunable parameters, even to the extreme. The main idea is to reduce the number of parameter within LoRA modules as much as possible while maintaining or even improving fine-tuning performance. This line of work is a sub-area of PEFT research, where similar ideas have been explored in the past (e.g., the work of Karimi Mahabadi et al. on Compacter architectures, or a very recent work concurrent to this one: LoRA-XS). To this end, the authors propose VB-LoRA which aims to use globally shared vector banks comprising 'constituent' sub-vectors that can be recombined into local 'delta' parameters (of the \Delta W matrix) that modify/adapt the weights of the original model. The VB-LoRA is then evaluated on GLUE with RoBERTa (for NLU), E2E with GPT-2 (for generation), and Llama 2 on MT-Bench (for instruction tuning) to show its benefits in terms of paramater efficiency and performance.

Strengths

- The work is well situated and shows excellent awareness of other work in this subarea which also partially inspired the proposed VB-LoRA method. The chosen baseline models are largely adequate and the only baseline missing (to the best of my knowledge) is LoRA-XS (which is concurrent work, so it couldn't have been included anyhow). - The method is well motivated from a conceptual level and it is also well described formally, in a succinct and convincing manner. The idea does resemble matrix factorisation and tensor product-inspired methods a lot, so I would like to see additional links to previous literature here. - I appreciate the fact that the authors aimed to provide a comprehensive evaluation over NLU, NLG as well as instruction-tuned models. However, this approach has traded some depth of evaluation for its breadth (and the evaluation is not entirely adequate and comprehensive - see under "Weaknesses" later). - A careful derivation related to the number of fine-tuned parameters across different methods is a very useful addition to the paper, given the fact that the parameter efficiency-performance balance is the main topic of the paper. - The paper is very clear and well written in general.

Weaknesses

- Evaluation is not comprehensive: 1) When it comes to instruction-tuned models, the main findings are based on a single model from a single family (Llama 2), fine-tuned on a single instruction tuning dataset and evaluated on a single evaluation dataset. This is simply enough to move from anecdotal to more generalisable findings, and experiments with (i) additional models, (ii) more instruction tuning datasets (e.g., Flan-v2, Tulu v2, there are more), and (iii) additional evaluation sets (e.g., MMLU, GSM, BBH, HumanEval) are required. 2) The same holds for NLU and NLG experiments. NLU is evaluated only on GLUE (where it's widely established that performance on GLUE is already saturated and considered 'superhuman'). For NLG, only experiments with a single model on a single benchmark are run. As I mentioned above, the experiments are broad, but stay quite shallow, and this has to change in the revised version. - The paper is very quantitatively driven, but it doesn't delve into providing clarifications and explanations on why certain quantitative phenomena/findings are observed: 1) What is the intuition behind VB-LoRA outperforming LoRA? How can we explain this? What makes VB-LoRA even more suitable performance-wise? Isn't it counter-intuitive that an extremely efficient model can outperform here? Are the tasks then too simple (with very low intrinsic dimensionality) as is the case with some of the standard GLUE tasks? 2) How can one fine-tune the optimal VB size (b) and dimensionality of subvectors within the VB (Table 5) - is this task specific or model specific? What would happen if one increases parameter budget a bit - can we expect even better performance or not? In what cases? 3) Are there any sub-vectors that get selected more often than some others? Why? Overall, while I'm quite happy regarding technical aspects of the work, the paper can be much improved in terms of evaluation/experiments and discussion of the key findings (some error analysis would be useful as well).

Questions

- Lines 180-182. "It’s worth mentioning that adding the multi-index information to the vector selection mechanism can make the TKAM model structure-aware, potentially yielding additional benefits." This is unclear to me and warrants further clarification/discussion. Could you write a short paragraph on how the multi-index information would be added and what type of structure would then TKAM learn (become aware of)? - Additional discussion on the similarities and differences between TKAM and standard (sparse and soft) MoE-s is required. Are there any MoE-style paradigms (which incur a higher parameter cost) that could have been used instead of TKAM? In general, the work doesn't really examine the choice of TKAM as a go-to routing method for the selection of sub-vectors. - Can the authors elaborate more on the similarities and differences between their work and the Compacter work (beyond the fact that Compacter focuses on bottleneck adapters while VB-LoRA focuses on LoRA)? Conceptually, the papers operate in the same space with similar ideas.

Rating

7

Confidence

4

Soundness

3

Presentation

4

Contribution

3

Limitations

There are limitations to this work that should be further elaborated (see also under 'Weaknesses'). The paper didn't explore the whole space of options when it comes to vector selection, VB size, VB content interpretation, and task dependency of findings, among other things. The work also stays within the realm of monolingual (i.e., English-only), mono-modal (i.e., text-only) and mono-PEFT (i.e., LoRA-only) contexts, and this should also be adequately signalled in the Limitations section.

Summary

The authors present a modified version of LoRA called VB-LoRA which is a highly parameter efficient fine-tuning method. It uses a vector bank to represent the model parameters as a composition of vectors. This vector bank is then used to select top-k vectors using the top-k softmax function which are thereby pooled and arranged to form the A and B low-rank matrices of LoRA. The top-k admixture module function selected is a differentiable function so the whole model is trained end-to-end making it more efficient for a particular task. Ultimately the authors show that such as divide-and-share approach results in an orders of magnitude more parameter efficient technique.

Strengths

1. The most significant contribution of this paper is to develop an LoRA-like adaptation technique that is orders of magnitude more parameter efficient than LoRA but at the same time maintaining comparable or better performance in all the tasks. VB-LoRA has the best average performance in all the tasks for every model. 2. The second most important aspect of VB-LoRA is that the number of parameters do not grow linearly with the model dimension (number of layers and model dimension). Making the 'k' of top-k much smaller than This is highly valuable as the model sizes are getting increasingly bigger by the day. 3. The usage of differentiable top-k softmax (TKAM - mentioned in eqn. 1) is a great idea as it enables the paper to be trained fully for a specific task. Also dicing the vector to sub vectors of same size was great for sharing among all layers. 4. The authors have tried to go to the extreme in making the LoRA parameter count efficient by dividing even the low-rank vectors of LoRA and pooling them from a common vector bank. 5. The authors also provide adequate supplementary information such as the detailed hyper-parameters and the hardware used which is highly appreciated as it is highly valuable for anyone who wants to replicate these results. 5. The authors also provide the exact code as a supplementary material which is a huge plus, especially as the paper consists of several moving parts. 6. The paper is presented in a very high quality way using multiple different concepts and giving proper references wherever needed. This fact would be appreciated by the readers as the paper uses several different concepts to come up with their design such as sparse admixture models, top-k gating module, canonical polyadic decomposition, etc.

Weaknesses

1. The parameter count of VB-LoRA is defined by: hb + 1.5LMr(d/b). The second term of this is still linearly dependent on the model dimensions (L and M). In the paper's experiments r<<b in which case the second is quite small and doesn't impact the growth in parameter by much. But there exist several adaptation tasks in which much larger ranks are needed. For example this paper (https://ieeexplore.ieee.org/abstract/document/10445894) shows that for domain adaptation large ranks (eg. 64) are much better than smaller ranks. In some of my other works for domain and language adaptation I have needed to use even larger ranks (200+) which are very much comparable to the value of b. In such cases, VB-LoRA's parameter requirement would be very much comparable to other methods. 2. Some parts of the paper are not entirely clear. For eg. in lines 166-169, the paper mentions that decomposed cumulative gradient parameters are more sensitive than the original model parameters during the training process. An intuition to why this is happening would be helpful. 3. In line 153, the paper mentions that by keeping k << h makes the learnt vectors highly specialized. However figure 3 shows that even with k's value as small as 2 and 3, the model updates almost all the vectors during the training process. Inherently this means that almost all the vectors have some activation and they are not very specialized. 4. Not a lot of details are provided for the virtual dimension 'b' and the results of table 5 are not explained. 5. nit: line 167: 'parameters'

Questions

1. Can you provide an intuition to your finding why the cumulative gradient parameters are more sensitive than than the original model parameters and how this affects adaptation? 2. Have you experimented with larger ranks and how do they look in terms of performance and parameter count? 3. Have you tried this technique on other tasks than what's listed in the paper? Also have you tried it on other architectures?

Rating

8

Confidence

4

Soundness

4

Presentation

4

Contribution

4

Limitations

1. In my view the authors have correctly mentioned that this paper has no larger societal impact beyond what LLMs may have. 2. Table-3 shows that the performance of LoRA may have improved with the changes in the GPT-4 model over time. But for the other tables the authors have used the old results for several competing models. 3. I agree with the rest of the answers to the NeurIPS paper checklist that the authors have provided.

The response clarifies some of my concerns

I would like to thank the authors for the provided response and the additional results, and I'm happy to increase my score in light of the provided new evidence.