Discussion 1/2
Thank you for your feedback and for the opportunity to clarify our stance further. Below, we address your concerns more explicitly under separate headings:
**Forget Quality vs Unlearning**
You raised a valid concern about the distinction between forget quality and unlearning, particularly in the context of differential privacy (DP). We acknowledge that our forget quality metric is not directly measuring the theoretical notion of approximate unlearning from the DP perspective. Instead, our work takes a practical approach, focusing on the empirical measurement of a model's ability to "forget" information related to the deleted samples. There is also a line of work that attempts to give more practical definitions to generative model privacy (e.g. copyright infringement), and our measure can be viewed as an instance of [1] with a special divergence.
The forget quality metric is designed to capture the similarity between the output distributions of the forget model (trained after deletion) and the retain model (trained without the deleted samples). While it may not strictly align with the DP-based definition of unlearning, it serves our purpose of evaluating the practical effectiveness of unlearning techniques for large language models (LLMs).
For your counter-example, since the two estimators (before and after deletion, forget model, and retain model respectively) are identical, they will have the same truth ratio. The KS test will suggest that they have the same distribution (of course, to be rigorous, these estimators have to be randomized, etc), hence good forget quality. So our method works here. *Importantly, our forget quality is not encouraging that the forget model should perform badly on the forget set, rather it should behave similarly to the retain model.*
To give you more intuition of the KS test, if the model is functionally equivalent to a retain model at “most” point in its input domain, the model “acts” equivalent to a perfectly unlearned model (in the sense of their output distributions, rather than the parameter distributions). This is precisely what the KS test aims to capture. While it can not exhaustively test the model at each and every point, the goal is to assess the fidelity of the “unlearned model” to the retain model, via a statistical KS test. To summarize, the KS test is a statistical test between the truth ratio of the retain and unlearn model on various question-answer pairs in the dataset.
Finally, we understand that you want us to be upfront about this distinction in the paper, and we agree with this aspect. We will make it a point to clarify this early in the draft to ensure our readers carry forward the correct message. We appreciate your detailed engagement on this issue!
This directly relates to the second point you raised. The goal of this work is to strive towards an “empirical notion” of unlearning that can scale. We use “forget quality” as defined by the above fidelity test, as a definition for this empirical notion, which is an all-encompassing umbrella term for both privacy and bias.
**Predictive Multiplicity**
We appreciate your concern about predictive multiplicity and the potential impact of different hyperparameters on the retain model's performance. In our work, we took measures to address this issue. As shown in **Appendix G**, we conducted extensive experiments to assess the robustness of the KS test to different training scenarios for the retain model.
Our findings indicate that the KS test is quite stable and provides consistent results across various hyperparameter settings for the retain model. We trained multiple retain models with different hyperparameters, such as varying degrees of dataset overlap and weight decay, and observed low variance in the outcomes of the KS test.
While we acknowledge the theoretical possibility of pathological cases where the retain model could negatively impact the evaluation, our empirical results suggest that the KS test is reliable in practical settings, even when the retain model is trained under different conditions.