BitDelta: Your Fine-Tune May Only Be Worth One Bit

Summary

This paper introduces BitDelta which quantizes the aggregated weight updates (the authors call it “delta”) to 1-bit after full fine-tuning. The paper claims that the approach has two applications: 1. First, it shows that the delta is highly redundant and 2. It is useful in the multi-client-single-server applications where the high precision model will be saved on the server and each client only store it’s own 1-bit delta. They show that in this scenario, the generation takes up to 10x memory reduction (and similar in latency improvement).

Strengths

1. The paper studies an important problem for fine-tuning LLMs with a new approach where they quantizes the delta to 1-bit, 2. The experiments are done on the most important models like LLaMa-2 and Mistral, 3. The paper provides a kernel for INT1xFP16 matrix multiplication.

Weaknesses

1, The author claim that BitDelta shows the potential redundancy of information added during fine-tuning. However, this is not a new finding and almost all PEFT approaches (for example LoRA) are based on this fact. 2. It seems that the paper completely missed the full fine-tuning costs and just measured the memory/latency of the serving step. However, I would suggest to have “apple to apple” comparisons and compare the fine-tuning+serving cost (including memory and runtime) of “full-fine tuning+1bit optimization+serving” against “fine-tuning+serving” in PEFT schemes (like LoRA). 3. In Table 6, the paper shows higher accuracy for FP+delta compared to GPTQ. Again, I would rather like to see memory Vs. accuracy tradeoff in such comparisons as a function of number of clients. The fact is FP+delta does not have the same memory as GPTQ (please current me if I am wrong). 4. It would be nice to have a performance model for the latency of the decoding as a function of client numbers. This is also missed and needs to be include as this is the main claim of the paper.

Questions

Please check the "weaknesses" section. I would be happy to discuss and change my score.

Rating

4

Confidence

3

Soundness

2

Presentation

3

Contribution

2

Limitations

I think the authors should define and present the limitations of the method more clearly.

Summary

Aiming at storage and serving overhead caused by multiple finetuned LLMs for various downstream tasks, this paper proposes a memory-friendly model compression method namely BitDelta which binaries the delta of each weight matrix and uses self-distillation to learn optimal scaling factors.

Strengths

BitDelta innovatively decomposes a fine-tuned model into its pretrained version and an additional weight matrix delta and then successfully binary delta to reduce memory overheads while preventing large performance degradation.

Weaknesses

The paper lacks innovation and has several points that do not hold up under scrutiny. For the first contribution, the paper proposes decomposing multiple fine-tuned models into a shared pretrained model and their respective deltas, and then binarizing the deltas. The specific decomposition method is not described in the paper. Based on experience, this can be understood as using LoRA to fine-tune LLMs and binarizing the learned low-rank matrices. This is not novel, as there are already existing low-bit model fine-tuning methods, such as Q-LoRA and QA-LoRA, which are more memory-friendly and do not require retaining an additional fp16 model. Regarding the second contribution, which involves using self-distillation to learn scaling factors, the paper's description is insufficient. For example, the initialization method of these factors is not well-explained. Furthermore, if the entire fp model's output is used to supervise the quantized model, the computational resource consumption is high. The paper could explore layer-wise optimization mechanisms to address this issue.

Questions

1) The author should describe the advantages of BitDelta compared to PEFT methods, such as QLoRA, QA-LoRA, and LoftQ. Because they don't require saving the fp16 pretrained model, which results in more efficient storage utilization. 2) More detailed description should be given, such as GPTQ+Δ in Table 6. Additionally, the performance of FP16+Δ being superior to 4-bit GPTQ and 2-bit Quip does not entirely prove that directly quantizing the base model is impractical, as INT8-RTN still shows better performance. If we consider 8-bit GPTQ, or other sota PTQ methods such as Omniquant, AWQ, it might also perform better while reducing memory usage by half. Therefore, it is necessary to provide additional arguments from other perspectives to explain why directly quantizing the base model is not preferable. Extensive additional experiments are not necessary, but it is important to clearly explain the aforementioned issues.

Rating

5

Confidence

5

Soundness

3

Presentation

3

Contribution

2

Limitations

The authors have addressed the limitations.

Summary

This paper introduces Bitdelta, a method that enables quantization with just 1 bit. The main idea is to compress the weight delta into a scalar and a binary matrix. The experimental results demonstrate that Bitdelta achieves better performance compared to other techniques.

Strengths

- Easy to read and well-structured. - The authors effectively demonstrate that their proposed technique, Bitdelta, performs well compared to existing quantization techniques. - It is expected that Bitdelta can be extended to areas such as multi-tenant serving systems.

Weaknesses

- A significant contribution of Bitdelta lies in its ability to reduce memory consumption through 1-bit quantization. However, the paper lacks detailed evidence to support this claim. It is necessary to compare memory consumption for each compression technique, but currently, only Bitdelta is shown. - It would be better for the paper to mention and explain the figures within the proper location of text. For example, Figure 1 and Figure 4 are included but not mentioned in the text. It is hard to understand without sufficient explanation. - Table 1 lacks information about the base models for Bitdelta and SVD. It should clearly specify whether these are based on Llama-7B or Llama-7B Chat.

Questions

- I am curious about the clear differences between Bitdelta and other compression technologies. For example, can Bitdelta, which adopts the Post-Training Quantization (PTQ) method, be used in combination with other PTQ techniques? Also, what happens if you combine Bitdelta with compression techniques like pruning? It would be beneficial to discuss the relationships between various compression technologies - (lines 165-167) The authors said that generation latency is proportional to the GPU memory used. How can this claim be proven? It would be helpful to mention references or provide supporting data - I think BitNet has a similar purpose with the design of 1-bit quantization. Comparing Bitdelta to BitNet is required

Rating

6

Confidence

4

Soundness

2

Presentation

4

Contribution

3

Limitations

I think the paper will be strengthened with the measurements on GPU (or memory) consumption are added

Summary

The paper proposes to quantize the weight delta of a fine-tuned LLM to 1-bit and observes that the model quality only drops a little. During the binarization step, it requires calibrating the scaling factor with a few hundreds of distillation steps. This is less than a full fine tuning. Evaluation show that the proposed method produces higher model quality than other post-training quantization techniques such as GPTQ and QuIP.

Strengths

The paper discovered a nice trade-off between fine-tuned model storage size and model quality. Trading 16x lower weight size by only storing the 1-bit delta for the limited quality drop as reported in the paper is impressive. Importantly, the binarization step has a near-post-training cost, instead of fine-tuning from the beginning. The paper did solid ablation studies on how important the scaling factor calibration is, which well described the effect of each component in the proposed method. The latency study is also appreciated.

Weaknesses

While requiring low storage size, the proposed method introduces an extra binary-float matmul during inference. Although it is indeed a special kernel and can have much lower inference time than the float matmul, the overhead will become more significant when the base model is low-precision.

Questions

1. Line 133 mentioned the scale distillation is robust to the choice of calibration dataset. Does it mean that it can also use synthetic data? It will be an improvement if the paper presents such an ablation. 2. Line 226: can the quantized delta be merged with the quantized base model? For example, by synchronizing the scaling factors.

Rating

7

Confidence

4

Soundness

3

Presentation

4

Contribution

3

Limitations

The paper does not have negative societal impact as far as the reviewer can tell.

Reply

Thanks for your answers. The authors claim that they do not present a fine-tuning scheme, but this rather a "serving" approach. I think they should highlight this as the main message of the paper and re-defining the evaluation approach. I would stick to my current score.

Thank the authors for response

Thank the authors for the response. I have read all replies and comments from other reviewers.