Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resources

Summary

The use of LoRA in FL is challenged by the heterogeneity of downstream tasks and available resources among clients. Traditional FL methods often use the smallest viable LoRA rank for all clients for aggregation compatibility, which makes it hard to capture the full diversity of client contributions and fully utilize ample client resources. To fully leverage heterogeneous client resources for enhancing the global model's generalization ability, this paper proposes FlexLoRA, which enables the mixture of diverse LoRA weights across individual clients to account for local client resources and task differences by allocating the highest feasible rank given a client's resource budget to ensure all clients contribute effectively regardless of resource capability, with heterogeneous aggregation and redistribution of weights through SVD. Extensive experiments and theoretical analysis verify the effectiveness and scalability of the proposed FlexLoRA.

Strengths

1. Exploring heterogeneous rather than homogeneous LoRA ranks across clients to fully utilize their resources is an imperative research direction. 2. Numerous experiments and theoretical analyses are illustrated to support the effectiveness of the proposed method.

Weaknesses

1. Numerous typos remain in this paper, including but not limited to: 1) "${Client_j}$" should be corrected to "${Client j}$" in Figure 2; 2) "Effect" should be corrected to "effect" in line 249; 3) "r=200 2" should be corrected to "r=200" in Table 6. 2. In contrast to HETLORA [8], the proposed FlexLoRA needs to perform SVD instead of redistributing it directly before redistributing the aggregated global LoRA weight to clients. This seems to be because it provides better initialization for the low-rank matrices B and A. However, the authors do not mention this point, and this practice has already been presented in FeDeRA [42], so this may not be considered one of this paper's contributions. 3. Some relevant work in this field is missing, including but not limited to: [1] Liping Yi, Han Yu, Gang Wang, Xiaoguang Liu, Xiaoxiao Li. FedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuning. arXiv preprint arXiv:2310.13283, 2023. [2] Shangchao Su, Bin Li, Xiangyang Xue. FedRA: A Random Allocation Strategy for Federated Tuning to Unleash the Power of Heterogeneous Clients. arXiv preprint arXiv:2310.13283, 2023.

Questions

1. The method described in Section 3 omits too many preliminaries and algorithmic details, which is reader-unfriendly.

Rating

5

Confidence

4

Soundness

3

Presentation

2

Contribution

3

Limitations

1. The proposed approach essentially fails into the area of federated fine-tuning of LLMs with parameter-efficient fine-tuning (PEFT) techniques. However, this paper does not discuss any limitations that may be imposed by PEFT especially LoRA used in this paper, e.g., PEFT methods sacrifice performance due to the parameter update is limited to a much lower-dimensional subspace, especially when data across clients are non-IID.

Summary

Under the framework of federated fine-tuning large model, this paper proposes a simple and effective LoRA aggregation method with different ranks, which mainly focuses on the problem of client resource heterogeneity. Specifically, the full-size LoRA is first obtained, and then after normal aggregation, it is decomposed into a matrix BA with the corresponding rank of each client by SVD. Extensive experiments demonstrate the effectiveness of the proposed approach when client resources are inconsistent.

Strengths

1. Different rank-based LoRA is directly matched to client resource inconsistency. The method of first multiplying into full-size LoRA and then decomposing into different ranks based on SVD is simple and effective, and has certain rationality. 2. The information loss of the parameters after SVD decomposition compared with the original global parameters is given in Section 3.4, and this method is generally reasonable.

Weaknesses

1. Does it increase the communication cost to convert matrix BA to full-size LoRA before uploading? 2. It seems that the configurations of ranks in Table 1 are not very diverse. Can the proposed method adapt to very diverse and more flexible configurations of ranks? 3. Different ranks seem to be specific to client resource heterogeneity, but do not seem to be particularly optimized for task heterogeneity? 4. While there are performance comparisons with the very related HETLoRA which is used to solve the resource heterogeneity problem, should comparison results with the rest of the state of the art FL methods also be given.

Questions

As shown in the weaknesses section, we won't add anything more here.

Rating

6

Confidence

4

Soundness

3

Presentation

2

Contribution

2

Limitations

This paper lacks a detailed discussion of the limitations and shortcomings of the proposed method.

Summary

This paper proposes FlexLora to tackle FL's heterogeneous resources and data distribution problem. FlexLora allows for dynamic adjustment of local Lora ranks by employing SVD for weight distribution, improving the global model’s generalization ability.

Strengths

1. The experiments are comprehensive. The experiments verified that the proposed method is efficient and improves the generalization. 2. The paper is well-written and easy to follow.

Weaknesses

1. This paper claims to follow a cross-device setting, but cross-device setting usually involves thousands of clients, while the client number in the experiment of this paper is much smaller. If the method can only be verified in a small-scale experiment, cross-device setting is not a very rigorous statement. Cross-silo setting might be more suitable for this paper. 2. One of the main contributions of this paper is allowing for dynamic adjustment of local Lora ranks in FL. Some dynamic/flexible Lora methods already exist in central learning. I wonder whether these methods can also be applied to tackle the heterogeneity of downstream tasks in FL. Related discussions or experiments are suggested here.

Questions

Please refer to weakness.

Rating

7

Confidence

2

Soundness

3

Presentation

3

Contribution

2

Limitations

The authors have discussed the limitations of the model and client scale. For the small client scale, claiming a cross-device setting is not so rigorous. Cross-silo setting might be more suitable.

Summary

This paper proposes a LoRA-based federated fine-tuning algorithm called FlexLoRA. FlexLoRA keeps full-size LoRA modules on the server, and decomposes them into heterogeneous rank LoRA modules according to the client task and capacity. The heterogeneous local LoRA modules will be transferred back to full-size and then aggregated on the server. FlexLoRA can make LoRA-based federated fine-tuning more flexible and easier to be implemented.

Strengths

(1) This paper uses SVD to solve the heterogeneous LoRA rank problem in FL, which looks valid and straightforward. The explanation of the proposed method is clear and easy to follow. (2) The paper provides sufficient scalability study. With slightly increased cost per round, FlexLoRA can convergence much faster than baselines.

Weaknesses

(1)The main concern is the privacy issue of FlexLoRA. Since the server needs to preset all the local LoRA ranks, the server needs to get access to some client information such as the amount of data, computation resources, ..., etc. Will this threaten the privacy of local data? (2)The experiment shows results of language model around 1B. Why larger models (e.g., llama) and benchmarks (e.g., mmlu, gsm8k) are not shown in the body part. There are some results in the appendix but may need more detailed discussion in the experiment section.

Questions

please see the weaknesses

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

N/A

Thank for the response

I very appreciate the authors' response. My concerns have been well addressed, and hence I am willing to raise my score.