Thank you for your continued engagement and for providing further insights. We appreciate the opportunity to clarify and discuss the points you have raised.
>Even if training a discriminator is not costly, users will not train an additional discriminator, as it does not aid their fine-tuning tasks. However, the method proposed by the author makes an additional assumption about user behavior, assuming that users train a LoRA while also training a discriminator, which is not the case in practice. Therefore, I say the application of this method is limited.
To clarify, our approach in fact simplifies the process for user and enables an effortless update of the user's model from the user's perspective.
One desired application setting of our approach is when a user provides their private data to a service provider for training and hosting a PEFT model. The service provider will train both the LoRA for task data and LoRA for discriminator and then delete the private data. There are no additional effort required on the user's side. When the base model needs to be deprecated and updated by a newer model, the provider can simply use the prepared discriminator for transferring the PEFT model without asking the client again for their private data. Thus, our approach simplifies the process for the user, as they now only need to provide their data once and be settled for all future updates (the discriminator only needs to be trained once even for multiple transfers).
A user could also train a PEFT model and discriminator on their side to avoid providing sensitive data, which, given the low overhead of the discriminator, is quite practical.
We acknowledge that our method is intended at all future users of PEFT models and will not apply to PEFT models prior to our method adoption. However, we believe it is a small limitation given the demonstrated success of our approach. We are happy to mention this in the limitation section. Thanks for suggesting!
>Reference [2] is not merely "unfiltered synthetic data." It employs model inversion techniques to invert a topic prompt, which is then fed into a pre-trained language model to produce in-distribution data, similar to your objective.
We used the term "unfiltered synthetic data" to refer that our approach and the approach in [2] are orthogonal and can be applied together. Our approach focuses on filtering of generated data, while the approach in [2] focuses on the generation itself. A straightforward combined approach is to use [2] to perform synthesis, and then apply our discriminator. We will cite [2] in our final paper and address this combination as a promising future direction.
>Model inversion is also a technique used to generate data by optimizing inputs/prompts so as to produce a target output. Could you compare model inversion and prompting generation?
Thank you for mentioning this topic. Model inversions are typically expensive and their operation requires backward passes for the inversion. For example, [2] trains a prompt generator alongside the student model using RL to guide the training of the prompt generator to prompts where student and teacher disagree. So each transfer would require such training and this might be prohibitively expensive. Our discriminator is one time and low cost, and any synthesis used in our method is forward only thanks to discriminator ability to filter useful samples.
Additionally, model inversion is orthogonal to our discriminator filtering approach as well, similar to [2]. One can easily use model inversion for initial generation, then apply our discriminator filtering on the generated samples. We will address this in the discussion of our final paper.