From Instance Training to Instruction Learning: Task Adapters Generation from Instructions

Large language models (LLMs) have acquired the ability to solve general tasks by utilizing instruction finetuning (IFT). However, IFT still relies heavily on instance training of extensive task data, which greatly limits the adaptability of LLMs to real-world scenarios where labeled task instances are scarce and broader task generalization becomes paramount. Contrary to LLMs, humans acquire skills and complete tasks not merely through repeated practice but also by understanding and following instructional guidelines. This paper is dedicated to simulating human learning to address the shortcomings of instance training, focusing on instruction learning to enhance cross-task generalization. Within this context, we introduce Task Adapters Generation from Instructions (TAGI), which automatically constructs the task-specific model in a parameter generation manner based on the given task instructions without retraining for unseen tasks. Specifically, we utilize knowledge distillation to enhance the consistency between TAGI developed through Learning with Instruction and task-specific models developed through Training with Instance, by aligning the labels, output logits, and adapter parameters between them. TAGI is endowed with cross-task generalization capabilities through a two-stage training process that includes hypernetwork pretraining and finetuning. We evaluate TAGI on the Super-Natural Instructions and P3 datasets. The experimental results demonstrate that TAGI can match or even outperform traditional meta-trained models and other hypernetwork models, while significantly reducing computational requirements.

Paper

References (37)

Scroll for more · 25 remaining

Similar papers

Peer review

Reviewer 2ySK4/10 · confidence 4/52024-06-30

Summary

This paper tackled zero-shot learning of LLMs to acquire cross-task generalization. The authors focused on the LoRA adapter, one of the methods used for LLMs for parameter-efficient fine-tuning. They generate task adapters by feeding the task instruction to the hypernetwork. Experimental results based on T5-LM-Adapt and T0 models clarified that the proposed method outperformed previous hypernetwork-based methods.

Strengths

1. The alignment of LLMs to task instructions is one of the most important capabilities of LLMs. The proposed method achieved comparable performance with the simple but effective standard fine-tuning baseline even with 39% of the inference cost.

Weaknesses

1. Because the proposed method underperformed the standard fine-tuning on the RougeL metric, the advantage of the proposed method is inference cost. However, the explanation with respect to computational efficiency is limited. I understood the low inference cost is caused by separating the self-attention for the task input x and the instruction i. Let L be the length of x and M be the length of i. Because the proposed method adds the cross-attention between x and i, the computational cost of the proposed method is L^2 + M^2 + LM, in which each term corresponds to the self-attention of x, that of i, and the cross–attention. However, the standard self-attention requires only (L+M)^2. I consider that the difference is small and cannot understand why the proposed method saves 61% inference cost compared to the self-attention. 2. This paper lacks the most important ablation study. Specifically, Table 5 lacks the ablation of LoRA weights. The low-efficiency cost discussed in the above item is caused by the separation of the self-attention. Therefore, the advantage of the main contribution of the proposed method, the LoRA weights generation from the hypernetwork, was not clarified from the experiments. I consider that replacing the standard self-attention with two separate self-attentions and a cross-attention does not worsen the performance largely if the pre-training is sufficient. 3. This paper lacks implementation details. More importantly, the proposed method seems to increase the number of parameters compared to the T0/Tk-Instruct baselines due to the addition of cross-attention and HyperNetwork. Even if the proposed method is superior in terms of time complexity (the number of FLOPS), the number of the parameters and the space complexity should be discussed.

Questions

See Weaknesses.

Rating

4

Confidence

4

Soundness

1

Presentation

1

Contribution

2

Limitations

1. This paper mentions the increased training cost instead of the reduced inference cost in the appendix. This is fiar discussion. 2. See the third weakness.

Authorsrebuttal2024-08-12

Dear Reviewer 2ySK: We would like to thank you again for your detailed reviews. We hope that we have satisfactorily addressed your concerns. The deadline for the rebuttal period is approaching, and we sincerely hope to receive your comments and suggestions soon. Please feel free to let us know if there are any further issues we can address. Respectfully, Authors

Reviewer 2ySK2024-08-12

Thanks for the Response

Thank you for offering such a thorough response to my concerns and additional results. I understood that TAGI is not comparable to Full-FT on the target task because TAGI is not fine-tuned on the target task. I raised the score accordingly. On the other hand, I have the following concerns: W1. regarding effectiveness (the RougeL score): In comparison with Tk-Instruct in Table 2, there was little improvement. Although TAGI outperformed T0 in Table 3, it may be due to the knowledge distillation from Tk-instruct because TAGI used Tk-instruct, which is a refined version of T0, as a teacher model. That is, TAGI would not be effective in comparison with the standard instruction tuning. W1. regarding efficiency (inference cost): I understood that there is an advantage when there are multiple samples. On the other hand, even with standard self-attention, it is possible to decode multiple samples efficiently by using cache KV if the model is decoder-type. L237 states `We limit our scope to encoder-decoder models for our experiment'. However, it is mis-leading to state that **the scope of the experiment is limited**. I consider that **the scope of the research (or, the usefulness of the proposed method) is limited** to the encoder-decoder-type and this should be discussed as the limitation. In general, I am concerned that the effectiveness of the proposed method does not improve the standard instruction tuning, and that the efficiency of the proposed method is limited to encoder-decoder type models. In other words, I am concerned that TAGI may provide neither effectiveness nor efficiency in instruction-tuning of the current mainstream decoder-type transformer.

Authorsrebuttal2024-08-12

Thank you for your follow-up comments and for acknowledging the clarifications provided in our response. ### Response to W1: > W1. regarding effectiveness (the RougeL score): In comparison with Tk-Instruct in Table 2, there was little improvement. Although TAGI outperformed T0 in Table 3, it may be due to the knowledge distillation from Tk-instruct because TAGI used Tk-instruct, which is a refined version of T0, as a teacher model. That is, TAGI would not be effective in comparison with the standard instruction tuning. In Table 2, our method outperforms similar hypernetwork-based approaches, surpassing the previous best method HINT by more than 1.5. Although the improvement in the 'Def' case for the traditional full fine-tuning method Tk-Instruct is modest, our approach uses fewer fine-tuning parameters, training resources, and less training time. In the 'Def + 2 Pos.' case, our method significantly outperforms Tk-Instruct, except for the 11B model. The results in Table 3 do not pertain to Tk-Instruct. Instead, we fully meta-trained a model, replacing 'metatrain,' which incorporates versions of two different models: T5-lm and T0. From Table 3, it is evident that our method surpasses existing results for meta-training, including multi-task standard instruction tuning and all TAGI results. Additionally, our training parameters and overhead are considerably reduced, leading to a 10% reduction in computational requirements for inference. Additionally, in the ablation study shown in Table 4, the T5-XL model achieves a score of 57.3 even without knowledge distillation (w/o L_kl), which surpasses the metatrain result of 53.1. ### Response to W2: > W2. regarding efficiency (inference cost): I understood that there is an advantage when there are multiple samples. On the other hand, even with standard self-attention, it is possible to decode multiple samples efficiently by using cache KV if the model is decoder-type. L237 states `We limit our scope to encoder-decoder models for our experiment'. However, it is misleading to state that the scope of the experiment is limited. I consider that the scope of the research (or, the usefulness of the proposed method) is limited to the encoder-decoder-type and this should be discussed as the limitation. Thank you for highlighting this issue. We acknowledge it as a limitation and have discussed it in Appendix B.1. Our approach aimed to compare with prior work, and therefore, we followed the settings of those studies. However, contemporary decoder-only models such as LLaMA3, QWen2, and other advanced models have already undergone extensive QA for Supervised Fine-Tuning (SFT), serving as robust baselines. Additionally, the continued improvement in model performance using efficient parameter fine-tuning methods like LoRA on decoder-only models suggests the feasibility of generating efficient fine-tuning modules via hypernetworks. Even though the use of KV-cache can reduce computation, it does not undermine our method's initial goal of encoding instructions only once to minimize computational load from the input side. In Table 2, we used the results from the HINT paper for GPT-2 XL (1.5B) and OPT (13B), which perform lower than encoder-decoder models of the same caliber. However, these results do not represent the most advanced decoder-only models available today. **Next, let's analyze the KV cache.** Here, we will let $l$ be the number of layers, $d$ the model hidden dimension, $h$ the number of heads, $k$ the size of the keys/values, and $s$ be the length of the sequence we want to save. We ignore biased terms for simplicity. For decoder-only models, if we want to cache the key/value pairs for a given sequence, we will store $2lhks$ values - a key and value for every head in every layer, for each item in the sequence. Assuming \($kh = d$\), the memory cost of decoder-only models is proportional to \($lsd$\). In the default TAGI settings, we store two main components: the processed instruction sequence, which consists of \(ds\) values (one vector per token), and the LoRA weights, totaling \($2 * 32 * l * h * k * 2$\) values (with a 32-rank LoRA for both key and value per layer per head). The total memory cost is therefore \($ds + 128 lhk$\). Note that the default LoRA rank is set to 32, but this parameter can be adjusted to manage memory costs. Assuming \($kh = d$\), the memory cost of TAGI is proportional to \($ds + ld$\). In conclusion, although decoder-only models can save computations through KV caching, our method only computes the instruction once, directly reducing the processing length at the input end. Therefore, it has better scalability in terms of sequence length (larger $s$) and model size (larger $d$, $l$).

Reviewer 2ySK2024-08-14

Thank you for your detailed response. I raised my score because my concern about the effectiveness is resolved. I consider that the additional descrtiption of the detailed setup of Metatrain the discussion compared to Metatrain would be benefitial. Also, I apprecieate it if the discussion on the limitation and advantage regarding the transformer architectures is added to the paper.

Authorsrebuttal2024-08-14

We are pleased to have addressed some of your concerns and appreciate the valuable feedback and improved score. We recognize that these are areas where our work needs to be revised and supplemented. We will continue to refine our paper and experiments based on the feedback, especially regarding the limitations.

Reviewer Wqw97/10 · confidence 3/52024-07-02

Summary

The paper addresses limitations of current instruction fine-tuning approaches for large language models, which rely heavily on instance training with extensive task data. This limits adaptability to real-world scenarios where labelled task instances are scarce and broader task generalisation is needed. The contributions of this work are listed as follows: 1. The paper introduces Task Adapters Generation from Instructions (TAGI), which automatically constructs task-specific models in a parameter generation manner based on given task instructions, without retraining for unseen tasks. 2. TAGI uses knowledge distillation to enhance consistency between models developed through "Learning with Instruction" and "Training with Instance" by aligning labels, output logits, and adapter parameters. 3. TAGI enables cross-task generalisation through a two-stage training process including hypernetwork pretraining and fine-tuning. 4. Experimental results on Super-Natural Instructions and P3 datasets show TAGI can match or outperform traditional meta-trained models and other hypernetwork models, while significantly reducing computational requirements. 5. TAGI demonstrates effectiveness in generating adapters for unseen tasks without additional parameter updating or gradient back-propagation, avoiding inefficiencies of repeatedly encoding instructions during inference. The authors state their approach "enables the model to achieve specified tasks and learn from instructions on how to address a category of problems" while reducing inference overhead and boosting generalisation performance.​​​​​​​​​​​​​​​​

Strengths

Overall, this work is of high quality, and its strengths from my perspective are listed below. 1. *Novel learning paradigm*: Beyond the traditional instance-based training, the authors propose a new paradigm, "learning with instructions", to mimic human learning by understanding and transforming task guidelines to adapters (parameters), which represents a shift from instance-based training to instruction-based learning. 2. *Novel problem reformulation*: This paper reframes the challenge of cross/multi-task generalisation by formulating it as a task-specific parameter generation problem. This addresses a key limitation in current approaches. 3. *Thorough comparisons and ablation studies*: The authors evaluate TAGI on two large-scale instruction datasets (SNI and P3), which provides a robust test of their method. The work also compares TAGI against a wide range of baselines, including methods based on zero-shot in-context learning and hypernetwork. The authors conduct detailed ablation studies to understand the contribution of each component in their model, strengthening the validity of their results. 4. *Significance on bridging human and machine learning*: This is the most interesting aspect of the work. By mimicking human instruction-based learning, this work contributions to the broader goal of developing AI systems that can learn more like humans.

Weaknesses

While the paper presents a novel approach with promising results, there are several areas where it could be improved: 1. *Limited model size exploration*: The authors primarily focus on models up to 3B parameters, with only limited experiments on an 11B model. This leaves open questions about how well TAGI scales to larger models, which are increasingly common in state-of-the-art NLP. The authors acknowledge this limitation, but could strengthen their work by solving the problem. 2. *Narrow focus on encoder-decoder models*: The paper exclusively uses encoder-decoder models (T5-LM and T0), citing their superior cross-task generalisation. However, this limits the generalisability of their findings. 3. *Limited analysis of instruction quality*: While the paper focuses on learning from instructions, there's little discussion on how the quality or format of instructions affects performance. 4. *Limited exploration of hyperparameter sensitivity*: While the authors provide some ablation studies, a more comprehensive analysis of hyperparameter sensitivity would strengthen the paper. This could include: a) Exploring the impact of different LoRA ranks beyond the chosen value of 32; b) Analysing how the size of the hypernetwork affects performance and efficiency; c) Investigating the trade-offs between pretraining steps and finetuning steps; Addressing these points would strengthen the paper's contributions and provide a more comprehensive understanding of TAGI's capabilities and limitations.

Questions

Please refer to my weakness section.

Rating

7

Confidence

3

Soundness

3

Presentation

2

Contribution

4

Limitations

Please refer to my weakness section.

Reviewer UQYV7/10 · confidence 4/52024-07-12

Summary

This work proposes a new learning paradigm to train large language models (LLMs) for its better task adaptation and generalization ability. Specifically, they propose the method called TAGI, which follows a two-staged teacher-student fashion by firstly learning a set of task-specific LORA weights and then utilizing the outputs from these to align and meta-learn the construction of unseen task adapter hypernetworks. The encoded task instructions will be transformed to the adapter hypernetwork weights for the downstream task adaptation. Such networks undergo generation distribution-based alignment training as well as task-specific training in wider ranges. TAGI is evaluated on the SNI multi-instruction tasks, demonstrating good performance against the baseline learning methods.

Strengths

- The proposed method is neat and cleverly integrates the hypernetwork style of training with the teacher-student alignment methods. - The demonstrated efficiency with yet other hypernetwork-based methods is intriguing. - The additional studies on ICL as well as self-ablation studies are appreciated. - The paper is quite easy to follow and well illustrated.

Weaknesses

- While I do appreciate the meta-train varying experiments, the distribution shifts of tasks in meta train or test datasets seem to be consistent (i.e., I did not find explicit evidence in the manuscript saying that some of the meta-test tasks are altered). Experiments on a few trials on the train-test splitting combinations are worth investigating to provide a more comprehensive analysis (with mean and variance). - Why is there no comparison between TAGI to prefix-based [1] methods? - Slightly minor: Particularly for the ICL, how would this work complement or benefit methods such as sample selection [2] or of example optimization style? - This is more like a suggestion: It would be good to also have a plotted analysis on the parameter size (can be presented in terms of percentage of the entire network size) of the generated hypernetworks against the corresponding performance. [1] Li, Xiang Lisa, and Percy Liang. "Prefix-tuning: Optimizing continuous prompts for generation." ACL 2021. [2] Gupta, Shivanshu, Matt Gardner, and Sameer Singh. "Coverage-based example selection for in-context learning." EMNLP-Findings 2023.

Questions

- What if you use significantly more positive examples to define the task and generate the hypernetworks?

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

- The authors address the limitations of the work quite thoroughly in the appendix.

Reviewer z6rG7/10 · confidence 3/52024-07-13

Summary

The authors introduce Task Adapters Generation from Instructions (TAGI), which converts instructions into task-specific adapters using a hypernetwork. They employ the Knowledge Distillation framework and a two-stage training process: first, hypernetwork pretraining on standard text pretraining data, followed by fine-tuning on meta-training tasks. They evaluate the proposed method on the Super-Natural Instructions and P3 datasets, demonstrating that TAGI can effectively generate adapters for unseen tasks.

Strengths

- The limitations of existing methods were clearly explained. - The writing was clear, making it easy to grasp the proposed methodology. - The concept figure was intricate and facilitated easy understanding. - An ablation study was conducted, demonstrating the validity of the proposed methodology's elements.

Weaknesses

- Following a method that assigns indices to the LoRA layer, the flexibility of the LoRA architecture structure is reduced. - This methodology seems limited to encoder-decoder structured LLMs.

Questions

- Please explain how $\lambda_1$ and $\lambda_2$ were tuned. - Could you provide more details on the process of updating LoRA parameters via $\mathcal{L}_{ins}$? - How were the tasks categorized? Please specify the criteria used.

Rating

7

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

Please refer to the Weakness section.

Reviewer UQYV2024-08-12

Thanks for the detailed rebuttal, I hereby retain my positive score.

Authorsrebuttal2024-08-13

We deeply appreciate you taking the time to thoroughly review our work and provide thoughtful feedback. We are pleased the additional experiments adequately addressed your questions. Thank you for your openness and willingness to reconsider based on new evidence. Your flexible and fair review process facilitates quality research!

Reviewer z6rG2024-08-13

Thanks for the rebuttal

Thank you for the detailed response. Most of my concerns have been resolved, and exploring whether the approach remains effective beyond encoder-decoder models could be an interesting direction for future work. I will keep my score as it is. Thanks.

Authorsrebuttal2024-08-14

Thanks for your suggestion! We are pleased the additional experiments adequately addressed your questions. We are currently expanding this work to models based on decoder-only architectures.

Authorsrebuttal2024-08-14

Overall Response to Reviews

**We appreciate all the reviewers for their constructive feedback and recognition of the contributions of our paper!** ### Strengths: 1. **Novelty**: Reviewer UQYV and Reviewer Wqw9 value the innovative approach of instruction-based learning, which emulates human learning by comprehending task guidelines and converting them into adapters (parameters). This marks a departure from instance-based training to instruction-based learning, treating all instances of a task collectively and thereby improving cross-task generalization. 2. **Sound Methodology and Strong Performance**: Reviewer z6rG and Reviewer UQYV note that the proposed TAGI method ingeniously combines the hypernetwork training style with the teacher-student alignment method in a concise manner. It validates the effectiveness of the method using large-scale datasets P3 and SNI and outperforms previous hypernetwork-based methods, while also having a clear inference advantage over the original instruction fine-tuning meta-training. 3. **Efficiency**: Reviewer 2ySK highlights that our proposed method achieved comparable performance with the simple but effective standard fine-tuning baseline even with 39% of the inference cost. 4. **Improved Generalization and Adequate Ablation**: Reviewer z6rG, Reviewer Wqw9 and Reviewer UQYV highlighted that converting instructions and task-related data into parameter-efficient modules enhances task generalization. Meanwhile, comprehensive ablation studies have demonstrated the effectiveness of each component of the method. ### Supplementary Materials: Once again, thank you to all reviewers for your constructive comments! We have added additional experiments and will provide further explanations: 1. How $\lambda_1$ and $\lambda_2$​ were tuned; (Reviewer z6rG) 2. Details on the process of updating LoRA parameters via $L_{ins}$ and the tasks categorized. (Reviewer z6rG) 3. Comparison between TAGI to prefix-based methods, LoRA-based methods, and TAGI from replacing the hypernetwork with LoRA (w/o Hypernetwork) (Reviewer UQYV, 2ySK) 4. Analysis of the parameter size against the performance. (Reviewer UQYV) 5. Comparisons of trainable parameters were conducted alongside additional theoretical analyses of inference complexity and space complexity. (Reviewer 2ySK) 6. More analysis of hyperparameters and instructions. (Reviewer Wqw9) 7. In the limitations section, we clarify that the current research is limited to models with encoder-decoder structures. (Reviewer Wqw9, z6rG)

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC