Summary
This paper introduces Dynamic Skill Adaptation (DSA), a framework that enables large language models (LLMs) to adapt to new and complex skills more effectively. Drawing inspiration from human teaching methodologies, DSA begins by creating a skill graph that breaks down complex skills into sub-skills, organizing them according to their dependencies. For each skill, DSA generates textbook-like explanations and exercise-style data, allowing LLMs to simulate human-like learning paths. Throughout the training, DSA dynamically fine-tunes the training data by reducing the emphasis on simpler examples, generating more challenging instances, and filtering out erroneous data. Experiments with models such as LLAMA and Mistral demonstrate DSA’s effectiveness in enhancing adaptation for tasks like math reasoning and social studies.
Strengths
* The writing is clear and easy to understand.
* Detailed ablation studies are provided to validate the impact of each component.
* The dynamic training approach appears novel, offering a simple yet effective method for data filtering.
Weaknesses
* All data collection steps involve using prompts for GPT-4; however, details on prompt design are lacking. I believe the prompt design could significantly impact performance, as the proposed method heavily depends on data collection.
* The method seems to implicitly distill domain-specific knowledge from GPT-4, but its performance still falls short of the teacher model (GPT-4). This raises a concern: why invest in costly API calls for data collection to create a specialized LLM that performs below the more general GPT-4? A more compelling approach might involve using Llama 2 to generate the data, thereby demonstrating that DSA can effectively improve LLMs' own domain-specific performance.
* Given the broad training data used by GPT-4, a more rigorous analysis of potential data leakage is warranted. Although the authors state that they conducted a sanity check to rule out exact matches between the test samples and training data, it is unlikely for GPT-4 to exactly replicate its training data verbatim. A more convincing approach would be to demonstrate that the exercises generated by GPT-4 do not yield the same answers as any questions in the test set. Besides, DSA models underperform compared to ChatGPT on the authors' custom Arithmetic task, yet outperform on the main tasks (pre-calculus and social studies). This discrepancy raises further concerns that data leakage may be influencing results, with DSA models potentially gaining an advantage by simply memorizing test data.
Questions
Q1: What are the proportions of $E_{easy}$, $E_{hard}$, $E_{error}$ and $E_{ambiguous}$ in each iteration? Is the initial error rate high or low? Is the majority of the data ambiguous, or can most be classified into a specific category? I would like to know more about how they change through the iterations. Additionally, there is a minor typo in Algorithm 1: $L_{ambiguous}$ should be $E_{ambiguous}$.
My main concerns lie in the weaknesses outlined above. These issues are significant, and without thorough clarification and detailed analysis of these points, it is challenging to assign a positive score.