Official Comment by Authors: Part 1
Thank you for reviewing our paper and providing insightful feedback. We're glad you find our work well-motivated and practical. We will address your concerns in the following points.
> **Q1: Regarding the technical innovation and contribution of FuseChat.**
**A1:** We thank the reviewer for the feedback regarding the technical contributions of our work. We would like to emphasize the uniqueness of FuseChat and clarify its distinctions from prior works such as FuseLLM and TIES from multiple perspectives:
**1. Distinction from FuseLLM**
**a. Motivation**
While FuseLLM emphasizes the fusion of multiple base LLMs through continual pre-training, FuseChat focuses on integrating diverse chat-oriented LLMs into a unified chat model via supervised fine-tuning. This difference in both training objectives and data makes FuseChat essential in the context of chat-focused LLMs. Moreover, our work extends beyond FuseLLM’s scope by fusing six distinct chat LLMs (as opposed to FuseLLM’s three base models), thereby demonstrating the scalability and depth of our methodology.
**b. Methodology**
While FuseLLM directly employs multi-teacher distillation to fuse multiple base LLMs, FuseChat employs a sophisticated fuse-and-merge approach, beginning with pairwise fusion and advancing to our SCE merging strategy. This new method is not only highly scalable and efficient but also better resolves knowledge conflicts in the parameter space. Simultaneously, it integrates the strengths of each source LLM with precision. By adopting this refined approach, FuseChat noticeably enhances the final model’s performance, distinguishing it from the techniques employed by FuseLLM.
**c. Scalability**
Another key strength of FuseChat lies in its plug-and-play approach for integrating new LLMs, which is more efficient than FuseLLM. Instead of combining distribution matrices from all source LLMs, FuseChat merges the distribution matrices of the new source LLM with a pivot LLM. This streamlined process reduces computational and storage costs, ensuring superior scalability as the number of LLMs increases.
**d. Experimental Validation**
Our experimental setup demonstrates the distinct focus of FuseChat. By fusing six varied chat LLMs (OpenChat-3.5-7B, Starling-LM-7B-alpha, NH2-SOLAR-10.7B, InternLM2-Chat-20B, Mixtral-8x7B-Instruct, and Qwen-1.5-Chat-72B), we validate FuseChat on AlpacaEval 2.0 and MT-Bench, assessing both instruction-following and conversational capabilities. This is in contrast to the base-model-focused experiments of FuseLLM and underscores the tailored contributions of FuseChat to the domain of chat LLM fusion.
**2. Distinction from the TIES merging method**
Our SCE merging strategy introduces considerable innovations compared to the TIES merging method:
**a. Automation and Precision**
Unlike TIES, which relies on manually tuned, model-level coefficients, our SCE automates the merging process by leveraging weight updates from a pivot LLM and computing matrix-level coefficients. This enables the fine-grained incorporation of diverse benefits across LLMs, which is difficult to achieve with manual hyperparameter tuning.
**b. Nuanced Parameter Adjustments**
In our specific context, where target LLMs are trained on identical datasets with relatively subtle parameter variations, SCE excels at capturing and preserving the distinctive advantages of each LLM through nuanced matrix-level parameter updates.
**c. Superior Performance**
Experimental results (e.g., Table 4) demonstrate that SCE outperforms baseline merging techniques including TIES within our framework, validating its efficacy and impact.
We will incorporate these detailed discussions into the revised manuscript to provide a clearer distinction of our work from previous approaches.