Generating Synthetic Datasets for Few-shot Prompt Tuning

Summary

This work tackles the problem that prompt tuning performs poorly in low-resource settings. It proposes a DawGen framework to generate synthetic data to augment prompt tuning and apply a gradient surgery approach to avoid conflicts from different data sources. Experiments shows that DawGen outperforms various baselines in low-resource prompt tuning.

Rating

6

Confidence

4

Ethics flag

1

Reasons to accept

1. The paper is clearly written and method is easy to implement. 2. The empirical results are sufficient to verify the effectiveness of the method.

Reasons to reject

1. More parameter efficient methods like LoRA are more commonly used in current LLMs and usually achieves better performance than prompt tuning. It would be better to examine the method's effectiveness on LoRA [1]. 2. Compared to the baselines (PPT [2], SPoT [3]), DawGen seems to introduce additional computation cost because it requires data generation for each downstream task while PPT and SPoT pre-train the prompt and applies it to various end tasks. This computational cost should be addressed. [1] LoRA: Low-Rank Adaptation of Large Language Models. [2] PPT: Pre-trained Prompt Tuning for Few-shot Learning. [3] SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer.

Questions to authors

Gradient surgery seems to be orthogonal to the prompt pre-training stages. How does it perform when combined with the baselines like PPT and SPoT?

Summary

The paper introduces a novel approach to enhance prompt tuning (PT) in few-shot learning settings by synthesizing task-specific labeled datasets using large language models (LLMs). The authors propose a method called Distribution-Aligned Weighted Generator Tuning (DawGen) to create in-distribution synthetic data, and they apply gradient surgery to manage gradient conflicts when training with both real and synthetic data. Their experiments across various datasets demonstrate that this method can outperform traditional fine-tuning techniques, presenting an efficient alternative when large labeled datasets are not available.

Rating

7

Confidence

3

Ethics flag

1

Reasons to accept

1. The paper proposes an original method for overcoming the data limitations of few-shot PT by generating synthetic data, which is significant for advancing the efficiency of using LLMs. 2. The experiments show that their method not only competes well with but occasionally outperforms much larger models trained with conventional techniques. 3. The paper includes extensive evaluations across several datasets, demonstrating the robustness and versatility of the proposed methods. 4. The paper is well-organized, making complex concepts accessible and clearly explaining the methodology and its significance.

Reasons to reject

1. While the method shows interesting results, there is not a unique method that has the best results. The best results are scattered between different methods. 2.Some parts of the methodology, particularly around the implementation of gradient surgery and the generation of synthetic data, could be more detailed to ensure reproducibility. 3. Are the superior results statistically significant.

Questions to authors

1.How robust is the proposed method to variations in the quality of synthetic data? Can the model's performance degrade significantly with lower-quality synthetic datasets? 2. How does the performance of DawGen scale with the size of the LLMs used for generating synthetic data? Would larger or smaller models significantly affect the outcomes?

Summary

This paper addresses the topic of prompt tuning, which typically requires large quantities of labelled training data and underperforms full-model fine-tuning in few-shot settings (by which the authors refer to low-resource settings rather than few-shot prompting). The authors present a method to create synthetic data using an LLM to do prompt tuning on and a method to fine-tune on the data, consisting of using gradient surgery when fine-tuning on a mix of the synthetic and real data. Their method consists in (i) training a soft prompt using a few-shot dataset to adapt an LLM to a given domain, (ii) using their method Distribution-Aligned Weighted GENerator tuning (DawGen) to align generated examples to those of the same class, and (iii) using label-conditional prompts and (iv) using gradient surgery when fine-tuning on a combination of synthetic and real data to account for possibly low quality data. They test their method using T5-large and FlanT5-large for 2 tasks (paraphrase detection and NLI) over 7 datasets, comparing to multiple other methods (fine-tuning, zero-shot, in-context learning, etc.). Their method gives results better than fine-tuning for some tasks and worse for others, and 18% improvement on average compared to basic prompt tuning for T5-large. In its current state, I found the paper unclear with respect to its main aim (more with respect to the writing of the paper - providing motivation for working with so few training examples for example) and would also have appreciated some more details concerning the method (see below). The paper is also lacking examples allowing me to review the quality of the produced synthetic data.

Rating

4

Confidence

3

Ethics flag

1

Reasons to accept

- It is interesting to have a comparison of different approaches to using LLMs and I appreciate the effort in combining methods to use the data in the best way

Reasons to reject

- Lack of general clarity in the paper about the main aim of the article. This is in my opinion linked to several things, starting from an unclear motivation concerning the use of “few-shot”, which in the context of prompting often refers to in-context learning, but here appears to refer to the case where we have few training examples (this could be made much clearer and non-ambiguous right from the start). On first reading, I was not sure why the authors were only using 16 examples per class of real data, which is so few. Also, why only choose 16 examples? If the main aim of the paper is to study whether prompt tuning can rival fine-tuning in low-resource settings, I think it would have been more informative to have the comparison at different numbers of training data examples in order to see the progression of the 2 approaches. With so few examples, fine-tuning must be hugely overfitting too. Is this a realistic setting? - In the main method, it was unclear to me what the input to the LLM looks like in order to produce synthetic training examples. Does it just consist of the label-conditional soft prompt and one of the 16 examples (for each class)? Also, I may have missed it, but how many examples were generated? - A major weakness in my opinion is the fact that no examples of generation data or predictions are given in the example. As a reviewer, this makes it impossible for me to judge the performance of the generation method and also whether the base LLM used is up to the job at all. The very low accuracy scores of the baseline on the binary tasks (below random) appear to show that the base model used may be inadequate.

Questions to authors

1. The meaning of “few-shot learning” was initially unclear to me (between using few-shot examples and using a very small set of training examples), particularly as the article is on the topic of LLMs, which typically use few-shot examples. I assume that the article wants to deal with very few training examples, but I only got this after rereading several times, and putting the article in the context of (Gao et al., 2022). 2. Synthetic data generation method: 
 a) It is Unclear to me what the input to the LLM is other than the label-conditioned soft prompt and the current example. 
b) What is the quality of the generated data like? There are currently no examples of generated examples in the article, making it impossible for me as a reviewer to judge the performance of the generation method. This is particularly relevant as the authors cite the inevitability of low quality data. 3. Missing details: a) Why was so little data used? 16 examples for each class? Is this a realistic setting? What about other amounts of data? b) Amount of synthetic data generated. How diverse is the data? Are there duplicates? c) In-context learning -> how many examples were used? d) What does the “number params” correspond to in Table 1? The models are the same, so does this refer to the number of tuned parameters? e) PPT appears to use T5-xxl and not T5. If so these results are not comparable. 4. In the related work, the authors mention that fine-tuning on a model on few examples (e.g. 16 samples per class) can lead to overfitting. What is the significance of 16 samples? i.e. why choose this number, which appears to me to be far too low to be realistic. 5. Verbaliser -> it would be good to make it clear what you mean by this in the heart of the text (it only became clear to me from the Appendix) 6. Very low scores of the baselines “Prompting” and “In-context”, especially given the binary nature of the tasks. Why do they do so poorly (Under random for some tasks)? Could this mean that the base model being used is not good enough and perhaps a better model should be used? ## Language, grammar, etc. - Few-shot real data -> real few-shot data - Comparable performance as -> comparable performance to - E.g., -> it is good practice to use a non-breaking space after E.g. to avoid linebreaks - Pertaining data -> Pretraining data - Which is inevitable to contain low-quality data -> which inevitably results in low quality data - Lester et al. (2021) converts -> Lester et al. (2021) convert - Taking sentence-pair classification as an example. Given -> Taking sentence-pair classification as an example, given

Response to the Authors' rebuttal

Thanks for your response. After reading your response, I think my rating is ok and I'm not change it.

Author-Reviewer Discussions

Dear reviewers, Thank you for your review of this submission. The authors have provided their rebuttal, and it is time to engage in discussions with them if you have additional questions. Please take the time to read the other reviews and update your own review as necessary. Reviewers 1fLs/T1e9: Even if you do not have further questions and/or choose to maintain your original score, please acknowledge that you have read the authors' rebuttal. Thank you! AC

Response to the Authors' rebuttal

Thanks for your response. After reading your response, I think my rating is ok and I'm not change it.