Towards Unbounded Machine Unlearning

Summary

This paper studies machine unlearning from an empirical perspective, proposes a scalable unlearning method SCRUB based on knowledge distillation, and evaluates the performance under different metrics along with other available unlearning approaches. Overall, this paper may provide insights and empirical metrics for designing other scalable unlearning methods in practice.

Strengths

Firstly, unlike other unlearning papers focusing primarily on user privacy, this paper summarizes three possible applications (removing biases, resolving confusion, and protecting user privacy) for unlearning algorithms to work in practice. Indeed, one should consider different metrics to evaluate the performance of unlearning algorithms under different scenarios. This is a valuable observation for designing other unlearning methods. The authors also show that previous state-of-the-art methods fail to obtain good performance on all tasks, which is exactly due to the fact that previous methods only focus on one criterion and thus cannot have universal good performance. Secondly, the proposed method SCRUB is based on a teacher-student model formulation, which has high scalability that can support large forget sets. Many previous unlearning methods, especially those having theoretical guarantees, would need some kind of sequentially unlearning process and repeatedly computing the Hessian matrix or Fisher information which leads to unavoidable high computational complexity. On the other hand, SCRUB treats the forget set as a whole and aims to have a consistent performance on the forget set. Lastly, this paper provides a comprehensive empirical evaluation to validate the superior performance of SCRUB. Moreover, for user privacy applications, it uses the latest LiRA membership inference attack accuracy as a metric and adapts the SCRUB method with a rewinding step, which has indeed proven to be helpful by simulations. The full code and documents are included in the SI for reproducibility. Overall this paper is well-written and easy to follow.

Weaknesses

Firstly, as the authors have pointed out in the paper, the idea of using knowledge distillation to achieve unlearning is not new, and the metrics, including using MIA accuracy to evaluate the unlearning performance are also not new. So my biggest concern for this paper is that the contribution might seem incremental. Also for the adaptation of the LiRA attacker, although the authors claim that "the first adaptation of LiRA for unlearning", but it is not clear why this type of adaptation is really needed. Is it because this adaptation can greatly increase the attack accuracy? Or is it technically challenging to apply LiRA directly within the framework? I would assume it is the first one, but some clarifications on these would make the paper clearer. Secondly, I am not sure if SCRUB can work well when the forget set is small but the training set is large. Consider the case where a model uses millions of data points for training, but we only want to unlearn one (or a very small portion) data point. This can happen frequently, especially in the UP case, where only a few users want to withdraw their data at the same time. Based on SCRUB+R, we would need to create a validation set of the same distribution as the forget set, which is basically sample another point from the same class as the validation set, and the goal is to make the model performance on forget point and validation point as close as possible. Since the choice of validation point could be any point from that class, such kind of requirement might lead to unstable training and the final model performance would largely depend on which point we choose. This is just an extreme example, but such kind of trend also appears in Figure 3, where the difference between SCRUB+R and Retrain becomes more significant when the forget set is smaller. However, despite these weaknesses exist, I think the observations, applications, and evaluation metrics proposed in the paper is still meaning for the unlearning community. So I would recommend boardline accept for the paper.

Questions

Here I summarize some questions from the weakness section. 1. Why the adaptation of LiRA is needed? Is it because allowing the attacker to know the unlearning algorithm can greatly increase the attack accuracy? 2. What would change in the unlearning algorithm when we only want to unlearn a very small portion of the training set? 3. What is the real difference between Removing Biases (RB) and Resolving Confusion (RC)? From the description, it seems that their goals are both not to predict the original labels of the forget set, and they share the same algorithm. I believe they can be both viewed as data cleaning for model training.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The authors have listed detailed limitations in the Appendix. First and the most important limitation would be it is hard to analyze SCRUB theoretically, as it always is in deep learning. However, when it comes to user privacy, sometimes a theoretical guarantee is required, which prevents the usage of SCRUB. The authors also mention the difficulty in training the min-max objective function, and how it will affect the model performance. Finally, the authors show cases where the rewinding procedure fails to prevent the MIA model to recognize the forget set, leading to privacy leakage in such cases.

Summary

The paper proposes three distinct goals to machine learning: removing bias, resolving class confusion from mislabelling of training data, and privacy through the right to be forgotten, and suggests that these goals require different technical approaches. The paper proposes an linearized multi-objective optimization problem (minimize loss on retained examples, minimize average KL divergence between softmax outputs to the original ones on retained examples, and maximize it on forget examples) with an alternating step optimizer. To limit the membership inference risks associated with ensuing high loss on forget examples, the paper proposes early stopping with a specially crafted validation set. The paper evaluates the algorithm for three goals on two datasets and using two network architectures.

Strengths

- S1. The proposed optimization objective is simple and seems to be more effective than a multitude of more complex approaches from prior work. - S2. Relatedly, the performance of the algorithm seems to be significantly more effective with multiple goals in mind. - S3. The paper delinates three different goals for unlearning that could need different technical approaches.

Weaknesses

- W1. *Impossible to evaluate the technical contributions without a detailed reading of the appendix.* The main body of the paper does not contain sufficient technical details to understand the contributions, deferring many crucial details to the appendix: the alternating step algorithm which is crucial to the contribution, training dynamics of the proposed objective that is inherent to the proposed optimization objective, the exact construction of the validation set for "rewinding", etc. Although the paper does not read bad, this is the reason I give the lowest score to the presentation rubric. It is not possible to judge the technical contributions without a reading of the appendix, which goes against the point of the page limit. - W2. *Insufficient evaluation in some aspects.* Although in many regards, the evaluation is comprehensive, e.g., with respect to the coverage of the methods from prior work, it is rather incomplete in other aspects. In particular: - We only see the results on two datasets and two models. - We do not necessarily know how significant is the performance improvement under optimization uncertainty. Such uncertainty might have non-trivial effect due to the unstable nature of the training dynamics of the proposed optimization objective. - The proposed optimization objective is simple and is only slightly different from the prior NegGrad work. I do not think this constitutes lack of novelty, especially as it seems that the proposed simple approach outperforms many complex ones. However, in this case, it is important to see ablation studies with respect to the design choices of this objective. For instance, why KL divergence in softmax, not loss for both retain and forget sets, or for one of the two sets? Are there any possible stability inducing terms that would get rid of periodic training dynamics? - Although I understand this is computationally expensive, especially if comparing to the retraining baseline, it would be good to see the performance in at least one large-scale dataset setting. It is important as machine unlearning seems most useful in such settings; otherwise, retraining might be good enough. In choosing the evaluation settings the paper cites prior work, but some of the prior work has evaluated, e.g., on Imagenet-1K.

Questions

---

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

1 poor

Contribution

3 good

Limitations

See W2.

Summary

The paper presents a Machine Unlearning method and shows its utility in Removing bias, Resolving confusion along with protecting user privacy. The experiments are conducted with different sets of evaluation metrics for each application. The latest Membership Inference Attack is adopted for Unlearning to evaluate the information leak in the unlearned model. The paper also stresses on relaxing the requirements of theoretical proofs for the bounds on the Machine Unlearning algorithms in general. This argued is due to the Unlearning methods utility in other applications than Data Privacy in ML.

Strengths

The following are some of the strength of the paper: 1. The paper shows the utility of Machine Unlearning in removing Bias and Resolving confusion. 2. The paper is well written and easily understandable. 3. Adoption of LiRA MIA attack is a good addition and to the evaluation of Unlearned Model. 4. Several experiments are conducted and comparison with some of the existing methods is discussed.

Weaknesses

1. Excessive Similarity with an existing paper – The main idea of this paper shares a lot of similarity with the following paper https://ojs.aaai.org/index.php/AAAI/article/view/25879/25651 (mentioned as Bad-T in the paper) In Page-6, Line 239-241, it is mentioned that “further differences with Bad-T is discussed in Appendix”. However, I could not find any such comparison. Since, there is a lot of similarity between the proposed method and Bad-T, it would have been appropriate to give a technical difference between the proposed and Bad-T method. But only the experimental results are compared in the Appendix. There are excessive similarities in the method described in Section 3.1 and Bad-T paper. The list of the similarities are: a. Page 3, Line 129-136. This is same as the Bad-T paper which also treats the original model as a teacher and the student selectively obeys the teacher for retain (Dr) and forget (Df) samples. Refer to the “Section: Proposed Method” in Bad-T Paper. The training object in Bad-T is also based on the KL-Divergence between the output distributions of the teacher and student model. b. Similar to Bad-T method, the teacher model is frozen here. Similar to Bad-T paper, the student is initialized with the weights of the teacher. Only difference is that the optimization is done alternatively for the retain (Dr) and forget (Df) samples. This is again quite similar to Bad-T optimization where the optimization for retain (Dr) samples are done using the competent teacher (original model) and optimization for the forget (Df) samples are done using the bad teacher which acts as a good proxy for the student model to move away from the original model. Particularly, if we take the variation of Bad-T where the bad teacher is just a random generator, wouldn’t that give a better robustness against MIA attacks? Because it is training the model predict random outputs instead of deliberately attempting to move the weights towards the opposite direction for forget samples (Df). c. The proposed method might be more prone to being unstable as already discussed in the limitation section. A comparison with Bad-T in terms of Stability was desired. d. Overall, it seems the entire method is almost similar to the Bad-T paper. Besides, the authors did not clearly mention the technical difference with Bad-T. Furthermore, as mentioned in the paper, Bad-T is significantly faster than the proposed method. In Machine Unlearning, speed is one of the most important factor. It is quite possible that the Bad-T method (or its other variants) might give similar results on all the experiments presented in the paper with better speed. 2. Inadequate Literature Study: In Page 2, Line 54-58, the authors explains that there is a lack of Machine Unlearning methods that are free from the assumption of the stability of SGD and can work with bigger datasets. In the literature study Section (Page 5) also, the authors primarily highlight the 3 papers by Golatkar et al. and derive their motivation for this paper. In past few years, there have been several papers that have highlighted the impracticality, inefficiency, and non-scalability issues present in the methods by Golatkar et al. Some of the papers that have shown Unlearning results without relying on the stability of SGD and other assumptions in Golatkar et al.: [1] “Amnesiac Machine Learning” [2] “Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks Using an Incompetent Teacher” [3] “Fast Yet Effective Machine Unlearning” Some more papers can be found here: https://github.com/jjbrophy47/machine_unlearning The idea of ‘unbound’ is not new as claimed in this paper. All these 3 papers have in fact shown good unlearning in ‘unbound’ setup. As far scalability is concerned, [3] have shown results on CIFAR-100 and VGGFACE-100 (class-level unlearning). The paper fail to mention these salient points in the literature study and in the experiments. 3. Considering the above mentioned points, the in the title of Section 2, the word ‘Unbound’ appears trivial. This Section gives a generic definition and discusses existing trade-offs in Machine Unlearning (MU). The content in this section doesn’t add anything significant to define ‘Unbound’. In fact, the Section title is little confusing. For example, in case of Privacy, we would like the MU method to be bound? Or in general unbounded methods are fine as they can be used to address multiple type of problems? 4. About SCRUB+R: Maybe this could have been part of the experiments and analysis section? It is just Early Stopping….

Questions

1. The authors could let me know their thoughts on Weakness-1 and Weakness-2. 2. Since, several existing Machine Unlearning methods have been presented in the ‘unbound’ setting (refer to weakness section). Maybe, it would be nice to discuss the idea of ‘unbound’ as a general category of methods and not something novel presented in this paper? 3. In Bad-T, there is a metric called Zero Retrain Forgetting (ZRF) which also aims to measure the generalization power of the unlearned model and not focus just on the Df and Dr accuracies. Is there any way the ZRF metric could be incorporated in this method? 4. I have few doubts about Removing Biases experiments – Are you unlearning a class or a subset of class or a set of random samples here? Also, it is unclear to me how the experiments on Removing biases are different from standard machine unlearning experiments. In fact, how exactly it is removing bias in the model? Could you elaborate? 5. Doubts about Resolving Confusion - I would like to know more about how exactly the confusion is resolved through unlearning?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

1. The experiments are performed only on CIFAR-10 which is pretty small-scale given the authors mention one of their motivation to design this method is to address the shortcomings of the existing Unlearning methods that work only on small datasets. 2. Several existing methods have already shown results on CIFAR-100 [1,2,3]. The proposed method could outperform some of these methods in 10-class dataset. It would be interesting to see how well the methods perform on 100-class dataset. 3. Sequential Unlearning? – A common use case of Machine Unlearning would require repeated unlearning over the period of time. The experiments on repeated unlearning is missing in the paper.

Summary

This paper introduces the methodology SCRUB, a teacher-student unlearning algorithm which personalises its forget metrics per application and dataset using. This method is demonstrated under three use cases: removing bias, resolving confusion, and user privacy. The authors support their theoretical claims with empirical experiments for each use case.

Strengths

The motivation of this paper is well-founded, and a major contribution to the field of unlearning. The paper is well-written and clear, with the contribution evident. The notation used throughout the paper is easy to follow. The empirical experiments have been well-chosen and SCRUB outperforms other baselines consistently.

Weaknesses

Some of the experiments seem a little simple (specifically the standard attack for the user privacy setting), but given the larger scope of the paper, this is understandable. If additional experiments could be added in this setting, it would greatly strengthen this specific use case.

Questions

- RB, RC, and UP should be defined again in the introduction - In Section 4, it would be nice to elaborate on why Bad-T performs poorly compared to SCRUB?

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

4 excellent

Limitations

Yes

Reviewer Response

I want to thank the authors for preparing the detailed responses and additional experiments. It is interesting to learn that the adaption of LiRA is critical in improving attack accuracy. The authors may want to emphasize this in the main text as well if the space allows. The authors have fully answered my questions and I do not have further questions.

Response to the Rebuttale

- Thank you for a detailed response to my queries and doubts. The additional comparisons with Bad-T and more results with CIFAR-100 are a welcome addition. - One important question remained unanswered about Sequential Unlearning. Does the SCRUB support repeated unlearning requests? and how the performance vary as more and more unlearning requests are made over a previously unlearned model? "...As explained in Section 3.2, it first must train “all the way”, until maximal error on the forget set is obtained. Then, based on that, it determines which previous checkpoint to use (“rewinding”). In addition, SCRUB+R uses an especially-selected subset of the validation set. Overall, our rewinding is not standard. It is a dedicated novel procedure, designed specifically to obtain an appropriate reference point for the forget error and avoid vulnerability to MIAs. Both the mechanism and underlying design principles are far from “just early stopping” - This is not clear in the paper. The authors should change the writing in the paper to convey this more clearly.