Machine unlearning is the problem of removing the effect of a subset of training data (the ''forget set'') from a trained model without damaging the model's utility e.g. to comply with users' requests to delete their data, or remove mislabeled, poisoned or otherwise problematic data. With unlearning research still being at its infancy, many fundamental open questions exist: Are there interpretable characteristics of forget sets that substantially affect the difficulty of the problem? How do these characteristics affect different state-of-the-art algorithms? With this paper, we present the first investigation aiming to answer these questions. We identify two key factors affecting unlearning difficulty and the performance of unlearning algorithms. Evaluation on forget sets that isolate these identified factors reveals previously-unknown behaviours of state-of-the-art algorithms that don't materialize on random forget sets. Based on our insights, we develop a framework coined Refined-Unlearning Meta-algorithm (RUM) that encompasses: (i) refining the forget set into homogenized subsets, according to different characteristics; and (ii) a meta-algorithm that employs existing algorithms to unlearn each subset and finally delivers a model that has unlearned the overall forget set. We find that RUM substantially improves top-performing unlearning algorithms. Overall, we view our work as an important step in (i) deepening our scientific understanding of unlearning and (ii) revealing new pathways to improving the state-of-the-art.
Paper
Similar papers
Peer review
Summary
This paper studies the difficulty of unlearning with respect to different unlearning sets. Specifically, the authors examine two factors, namely entanglement between forget and retain sets, and memorization. The authors further propose a framework called RUM which first divides the forget set into different subsets according to the memorization score and performs different unlearning algorithms to unlearn each subset.
Strengths
I agree that studying the difficulty of unlearning different data (subset) is a very important problem and would help us evaluate current approximate unlearning algorithms. I generally enjoyed reading the paper, especificallly Section 4, where the 2 factors are nicely chosen and the experiments (figure 1) well demonstrate their effects. Specifically, the memorization part aligns with the study of data attribution and matches my intuition for approaching this problem.
Weaknesses
I will merge the weaknesses and my questions in this part such that the authors can easily address each point during the rebuttal: **[Memorization]**: according to equation (1) and Section 4.2, it appears to me the authors perform "leave-one-out" experiments, where they train a model without a target data sample and calculate the memorization score in (1). This is repeated for every training data point (50000 times for CIFAR-10) and it seems quite expensive (if training until convergence). My questions here are: - Did you use any approximation in your experiments? How many epochs did you train? Can you show the computational cost? - In Section 6, the refinement step is also performed based on memorization scores. Thus I believe the calculation of memorization score should be also considered for "efficiency" in Section 7? Given a large model (e.g., LLMs), and a large forgetting set, this process would be extremely expensive and may decrease the practicality of RUM. **[MIA and ToW]**: The authors use ToW as the only evaluation in Section 4 and apply both MIA and ToW for Section 5. It is unclear to me why they omit MIA for evaluating unlearning difficulty. Is it because MIA may be a biased evaluation? Sequentially, for step 3 in RUM, will the selection of optimal unlearning algorithm be affected by considering MIA as well? **[Transferability]**: RUM relies on the observation in Section 4 to select optimal unlearning algorithms. This comes handy as Section 4 and 6 both consider the same datasets. However, in practical scenarios, this is not possible and one needs to rely on the assumption that one unlearning algorithm is optimal across different datasets and model architectures. Is this assumption true? **[Title and Section 4]**: The title of this paper and name of Section 4 "what makes unlearning hard" is misleading to me. This presentation suggests that the authors are studying: (1) unlearning is a hard problem; (2) why it is hard. However this is not true and they actually studied "the difficulty of unlearning regarding choice of forgetting sets". Specifically, the memorization subsection actually suggests different behaviors for different unlearning methods. I would suggest a more accurate delivery such as "on the difficulty of unlearning...". **Overall**, I like the problem of interest in this paper and I appreciate the takeaways from Section 4. I think the observations help the community understand unlearning better. However, I don't believe RUM is a super practical framework based on the tremendous effort needed to perform refinement and the unknown behavior of transferability. If any of my speculations are wrong I would love to hear back from the authors.
Questions
See above.
Rating
6
Confidence
3
Soundness
2
Presentation
3
Contribution
3
Limitations
Limitations are addressed.
Summary
This paper looks at measurable qualities of forget/retain sets in machine unlearning contexts, which can make it difficult to 1) unlearn the forget set at all 2) unlearn the forget set without also unlearning the retain set. The paper puts forward well-reasoned metrics that get at entanglement/homogeneity in the embedding space for the forget and retain set, and also examine memorization as a possible factor for complicating machine unlearning. Based on the lessons from this, the paper suggests a meta-learning algorithm to figure out how to use existing unlearning algorithms to better unlearn the forget set.
Strengths
This paper studies a very sensible set of questions with respect to qualities of the forget set that could make machine unlearning hard. The results (on the motivation side) are arguably expected, but someone needed to actually do the work to show this, and I'm really glad this paper did so (and did so very comprehensively, with respect to entanglement and memorization, and the ToW metric). Doing this / using these (perhaps expected) lessons to motivate the meta-learning RUM algorithm is also very principled, and also makes for a really tight paper. Without doing the first half fully, it would be hard to understand why the second half/ proposed algorithm is so well-motivated.
Weaknesses
Overall, I think this paper makes excellent contributions to unlearning science. There is one thing I would have liked to see addressed in more detail in the main paper, and I have some nit comments (that do not affect my score) that I think would improve readability. I organize my response here according to these distinctions, to (hopefully) make it clear which things could possibly lead to improvements in my score. **Affects my score, engaging with this could lead to me increasing my score** Given how expensive unlearning methods tend to be still, it would be great to see more discussion about efficiency. It is not clear to me entirely from the main paper if there are significant associated costs of using this meta-learning approach. These costs might be entirely worth the benefits presented (depending on the context of use, of course), but it would be really useful to include more on this in the main paper. However, I also note that this paper packs in so much already in the page count, that I understand why some things had to be cut. But this also led to other problems (which I understand are in tension with what I'm saying about including more details about efficiency): the quantity of presented results for a paper of this length has demanded really small (often difficult to read and process) tables and figures. The reason this is important is not just about readability. It takes a lot more time as a reader to unpack and verify the results in relation to the claims being made. **Nit comments, do not have an impact on my score** Math readability: Using different symbol classes for things of different types could help readability of the definitions a lot in this paper. For example, curly braces are used here for both algorithms and sets of datapoints. A simple improvement would be to use visual cues in the format to show they're different (not just with the chosen letter). For example, using bolded capitalization for a matrix of examples. Doing this can also make editing/finding typos more noticeable (e.g., line 60 has an $S$ for the forget set that, with the current style/format, should be $\mathcal{S}$). Please explicitly define $e$ in the follow-on to Definition 2.1
Questions
See weaknesses -- It would be great to understand more about efficiency and if there are interesting trade-offs here with respect to the proposed method
Rating
7
Confidence
4
Soundness
4
Presentation
2
Contribution
3
Limitations
See above re: efficiency analysis
Summary
This paper studies how two properties of “forget sets” make machine unlearning hard: similarity/entanglement between the forget set and an accompanying retain set, and a memorization score for the forget set. The authors hypothesize and experimentally confirm that (1) high entanglement makes unlearning harder, likely due to it being hard to learn a model that forgets data that is so close to retain data, and that (2) more strongly memorized data is harder to forget, because models that memorize such data tend to be quite different from models than models that never see that data. Building on these observations, the authors propose a new algorithm for unlearning that first rank-orders forget data in terms of memorization scores, then selects an unlearning algorithm from a pool of algorithms that achieves the best ToW scores on data with the relevant level of memorization (as determined by previous analysis experiments). Results suggest that sequential forgetting based on forget set memorization achieves immediate improvements over whole-set forgetting, and that leveraging unlearning algorithms with good known ToW scores also improves results.
Strengths
- Very important: The paper discovers a relationship between two retain set properties and unlearning difficulty, and leverages this relationship to improve unlearning performance on some metrics in a setting where these properties can be measured for forget sets. These are interesting findings in their own right, and provide a good starting point for follow up work. - Important: Experiments are straightforward and easy to understand, with only a few exceptions. - Important: The paper is very well written and enjoyable to read. - Important: The paper is well situated in the literature by the introduction, preliminaries, and related work sections. The connection to work on forgetting during training is interesting. - Of some importance: The paper contains some individual observations that are interesting, though not well supported by the remainder of the analysis. For example, that “operating sequentially on homogenized subsets according to memorization can boost the performance of unlearning algorithms” and that “a key observation from our results is that we can do well by actually doing nothing on a subset of the forget set”.
Weaknesses
- Important: The ToW metric seems very reasonable at first glance but has some worrying issues on further thought. Mainly, the fact that this metric compares average accuracies across a data distribution is problematic. Consider that two models could disagree on many individual data predictions while achieving the same overall accuracy. Thus, an unlearning algorithm could yield a very different model from $\theta^r$ while still achieving good ToW. I like that the metric balances three different desirable quantities, similar to a harmonic mean, but it seems better to balance existing, accepted metrics from the literature, like empirical measurements of divergence between unlearning/original model predictions, MIA, extraction attack success, and accuracy on retain and held-out test sets. Analysis in the paper is only conducted in terms of ToW (Sec 4). - Important: Results on MIA could be stronger. It appears that the refinement step accounts for most/all of the gains here, which is basically unexpected given the previous analysis in the paper. Improvements on ToW are positive, which would be good for the paper, if I didn’t have some skepticism around the ToW metric described previously. - Important: Conducting all experiments with ResNet models on CIFAR datasets limits the generalizability of the results. While a breadth of unlearning methods are considered, I think evaluations should be conducted with other datasets and architectures too to ensure generalizability of the results. - Of some importance: This is a personal opinion: I find the focus on unlearning in the differential privacy vein to be the wrong approach for many unlearning settings. I think it makes sense for users requesting their data not be used in training, and for model developers assuring approximate compliance with this request. I think it doesn’t make sense for unlearning private or dangerous information from models. The paper mentions extraction attacks once in the related work section, but extractability seems like it should be the core metric for unlearning to me. We either want a model to get a datapoint correct or get the datapoint wrong. Whether or not it was used for training seems inconsequential much of the time (one exception is when model developers owe people money for using their data). Reading this paper, this tension appears often. For instance, lines 242-246 reveal that unlearning algorithms can get worse ToW scores when they lower accuracy on a forget set too much, echoed in L.252-253 (cf. “overforgetting”). How can you overforget a forget set? I don’t like this problem formulation. I prefer defining forget sets as data where we want extraction attacks to fail, with retain sets being data where we want high model accuracy, and whether MIAs work on retain data is a matter of user data *access* (not privacy per se, since low-mem data might be relabeled accurately by models with high confidence, even when users do not grant access for training). All this said, I recognize that DP-based definitions for privacy remain popular in unlearning research, so I factor this point into my final score with less weight.
Questions
- Since the retain set is like training data, do you think it would be better to exclude it from ToW? - Entanglement score: there might be variance in a direction that seems irrelevant to whether the two distributions overlap. I feel that some MMD or classifier-based score could be better here. What do you think? - In Fig1b, why is Original so low for medium and high memorization retain sets? - typo: ln171: will cause accidentally erasing
Rating
5
Confidence
3
Soundness
2
Presentation
4
Contribution
2
Limitations
I think it is becoming increasingly important to evaluate methods on settings beyond CIFAR+ResNet. This setting is becoming a bit dated compared to current applications of CV or other ML methods. Of course you can’t evaluate on everything, but if the experimental setting is this specific, it could be noted. On broader impacts, a current (negative) dual use scenario for unlearning techniques is to remove the influence of important safety data from finetuned models, e.g. https://arxiv.org/pdf/2310.03693. I think unlearning work should generally be aware of this scenario.
Additional reply to rebuttal
>The reviewer’s point about looking at disagreements at the example level rather than average accuracy is a great one…and we see the same trends emerging according to both average accuracies and example-level disagreements, we feel there is no issues of “bias”; we in fact view this agreement of trends as evidence for the robustness of conclusions one would draw when using our contributions. Curious to hear the reviewer’s thoughts on this. Sure, so it seems like we agree the example level metric would look very different from the dataset level metric. In the limit, accuracy might not change at all while the two models might disagree on most examples, by virtue of trading off similar amounts of false positives vs false negatives. So, the metric is flawed at its core. It is very possible that two models get similar performance while producing very different predictive distributions. In fact this is common for two comparable NNets trained on a dataset, when the models don’t get near 100% accuracy. The metric would suggest that these models are indistinguishable, when this is not the case. The fact that, for the few models and datasets examined in this paper, this bias does not seem to exceed 15% (a large number) is not comforting to me. We know in theory it’s wrong. We should just change it. Note that this example-level disagreement can still become a scalar metric. It being a scalar metric is not the crux here. >We think the comparison that makes sense is to compare L1-sparse vanilla against L1-sparse RUM. Maybe accidentally the reviewer looked at the wrong rows in the table? (comparing L1-sparse vanilla against NegGrad+ RUM, but please correct us if we misunderstood your comparison). Well, it’s not a “wrong” comparison. There are two ways of doing the comparison. You could control for the underlying method (L1-sparse vs NegGrad) and assess the effect of RUM across such settings. Or you could ask, how good can RUM be across methods, compared to how well we can do across methods without RUM. Two different, valid comparisons. And for what it’s worth, always restricting to within-method comparisons suffers from the risk of multiple testing (https://en.wikipedia.org/wiki/Bonferroni_correction). > In brief, yes they are statistically significant. For L1-sparse in Table 1, the MIA gap falls from 0.089 to 0.033. Specifically, we computed the paired t-test to derive the CIs for the MIA gaps. For the MIA gap for (L1-sparse-vanilla - Retrain) the 95% CI is 0.089 +/- 0.032. For the MIA gap for (L1-sparse-RUM - Retrain) the 95% CI is 0.033 +/- 0.003. Great! I would recommend including this in the paper. In short, I am willing to believe there is an improvement from RUM, on the order of 3-5 points MIA. > The factors explaining difficulty (that we discover through analyses in Section 4) are what informs both the “refinement”, Ok, this is a good point. Doing the subsetting/ordering depends on identifying the factors in the first place. ___ Thanks for the proposed changes. I think they seem adequate. I would overhaul the metric, but if it’s the case that a reader immediately sees a footnote or appendix reference clarifying the average vs. example level difference (and results are also provided in the appendix with an example level metric), that’s ok. I leave it to the authors to include this discussion in the revised manuscript, and I will raise my score to Borderline Accept.
Dear reviewer 3Xcb Thank you for your comments. We assure you we will provide the additional discussion on the issues pertaining to example-level performance and our relevant results. We view this as an issue going beyond our paper - an important issue for future related work to consider going forward, thus affecting the community's research effort. Thanks!
Summary
This paper investigates the characteristics of forget data for machine unlearning algorithms and demonstrates their impact on the effectiveness of various unlearning methods. The paper further proposes refining the forget set into subsets based on data characteristics and selecting suitable unlearning algorithms for each subset. The experimental results show that this ensemble method outperforms single baselines.
Strengths
* The identification of properties in forgetting data is valuable for machine unlearning. The idea of adapting algorithms to the properties of specific data could benefit future algorithm design. * The proposed refine-based method is simple yet effective, as demonstrated by the experiments.
Weaknesses
* Despite the significance of the insights about sample embeddings, they appear somewhat limited to classification models, which typically have an embedding space to group different samples. While this may be beyond the scope of this paper, it would be valuable to explore extending these investigations to generative models, such as diffusion models and transformer-based large language models (LLMs). * The experiments were conducted only on two CIFAR datasets, both involving image classification tasks. Including other tasks or datasets, such as text sentiment or topic classification, would make the paper more comprehensive. * Although the paper claims the RUM framework is a meta-algorithm framework, the selection of unlearning algorithms for each subset is now based on empirical investigation and human-designed heuristic, which limits the generalizability of this method. minor: I would recommend refining Figure 2, as some text is misaligned and mixed with the box boundaries.
Questions
* What is the model accuracy before unlearning in experiments? * What's the criteria to group samples into low/medium/high partitions?
Rating
5
Confidence
3
Soundness
3
Presentation
2
Contribution
3
Limitations
Authors discussed the limitations.
Reply to rebuttal
Thanks for the response! Replies below. > **Re: Weaknesses of ToW**, we discern two interesting issues raised: 1) using average accuracies, 2) that it might be preferable to base ToW on another metric like MIA. Thanks for the response to (1). I would recommend adjusting the core metric to avoid any possible bias in results. If I’m reading the response correctly, it seems like a measure of bias could be as high as 15%. For (2), I think ToW-MIA is a good idea and appreciate the additional results. > We do not see why it is “unexpected” to see that, for MIA, the refinement step plays a bigger role. This is actually a nice finding, which may lead to further insights. To clarify, I meant unexpected here as in, the paper is focused on how two factors could explain unlearning difficulty, and then the results show that a third and seemingly arbitrary factor (the ordering of data into subsets based on memorization scores) plays the biggest explanatory role. The bulk of the paper does not concern data ordering. Why is data ordering important for unlearning? A whole different paper could be written on this (the paper points to [5] for relevant work). > Table 1 in the paper clearly shows that RUM introduces strong MIA gains! For any algorithm there, the RUM version has much better MIA (and ToW). We are thrilled with this result (and frankly surprised to read this criticism). To clarify, I was looking at MIA falling from 0.089 (L1 vanilla) to 0.059 (RUM). Is this a strong gain (and is it statistically significant)? Is this the comparison I should be looking at? > **Re: limited to ResNet and CIFAR**. We address this point by adding an additional dataset and architecture. Adding VGG-16 with Tiny ImageNet is an improvement >….MIAs are a common and convenient tool for measuring “leakage” and success of this type of unlearning. Overall, these are highly debated topics, with subjective opinions, and without consensus. While work is needed to settle these debates, we hope the reviewer agrees beyond doubt that our problem setting is widely considered a valuable one. I agree. It’s a minor point I wanted to make. I suspect that as the public learns more about ML + data privacy/deletion, there will be a shift towards extraction attack metrics over DP metrics, but I can’t deny that this is a legitimate and longstanding problem setting. > **Re: “how can you overforget a forget set?”** Following from above, our goal is to produce an unlearned model “similar” to that of retrain-from-scratch. So its accuracy on the forget set should be just as low as that of retraining, but no lower. We use the term “overforgetting” to describe the situation where the unlearned model causes a forget set accuracy lower than that of retrain. This is problematic as it can lead to vulnerability to MIA (due to noticeable differences from retraining). Thanks for the clarification. The argument makes sense to me. I get the feeling the terminology in the field (not just in the paper) could be tightened up a bit, because it is a little unintuitive to speak of “overforgetting” a forget set. (Forget sets should be forgotten!) > **Re: should we exclude retain set from ToW?** The retain set is important. For instance... Right, that makes sense. > We experimented with MMD and find there exists a (negative) correlation between ES and MMD scores. Thanks! > **Re: “why is original so low for medium and high memorization retain sets”**...Please let us know if this clarification helps. Got it, thanks! --- Based on the above discussion, I am prepared to increase my score if the authors can (1) reformulate the ToW metric to measure example-level changes in model predictions and not changes in average accuracy alone, and (2) clarify the strength of evidence there is for improved MIA. I would still point out that there is something confusing to me about this paper regarding the refinement step. The analysis in the paper focuses on memorization and entanglement, not data ordering. But then data ordering turns out to be very important for the improvements in unlearning in the proposed method.
Answers to new questions of reviewer 3Cxb
**“Re: …adjusting the core metric to avoid … bias. … it seems like a measure of bias could be as high as 15%.”** We created ToW as a reasonable metric that holistically captures a notion of "difficulty" in a scalar, **to allow for efficient analyses and investigations into unlearning difficulty**. The reviewer’s point about looking at disagreements at the example level rather than average accuracy is a great one! As far as our goals were concerned with respect to using ToW as a tool to measure unlearning difficulty, we actually see the same trends with our initial framing of ToW compared to this example-based disagreement score! Concretely, for the vast majority of unlearning algorithms, we find according to ToW that low mem forget sets are easier. Correspondingly, we also observe that for low mem examples, the disagreement rate is also lower. On the other hand, higher mem examples are harder to forget, as shown by average accuracy (in original ToW) and indeed the disagreement rate is also higher. Given that our goal is to study trends, and we see the same trends emerging according to both average accuracies and example-level disagreements, we feel there is no issues of “bias”; we in fact view this agreement of trends as evidence for the robustness of conclusions one would draw when using our contributions. Curious to hear the reviewer’s thoughts on this. **Re: “To clarify, I was looking at MIA falling from 0.089 (L1 vanilla) to 0.059 (RUM). Is this a strong gain (and is it statistically significant)? Is this the comparison I should be looking at?”** We think the comparison that makes sense is to compare L1-sparse vanilla against L1-sparse RUM. Maybe accidentally the reviewer looked at the wrong rows in the table? (comparing L1-sparse vanilla against NegGrad+ RUM, but please correct us if we misunderstood your comparison). In brief, yes they are statistically significant. For L1-sparse in Table 1, the MIA gap falls from 0.089 to 0.033. Specifically, we computed the paired t-test to derive the CIs for the MIA gaps. For the MIA gap for (L1-sparse-vanilla - Retrain) the 95% CI is 0.089 +/- 0.032. For the MIA gap for (L1-sparse-RUM - Retrain) the 95% CI is 0.033 +/- 0.003. **Re: “the paper is focused on how two factors could explain unlearning difficulty”, “a seemingly arbitrary factor (the ordering of data [...]) plays the biggest explanatory role.” and importance of data ordering.** Thank you for the discussion! What we show with RUM is that operating on **homogenized subsets of the forget set** (obtained in the refinement step) yields big gains (over random subsets as in our Shuffle baseline), and further, that pairing each such subset with the most suitable unlearning algorithm yields further gains. The factors explaining difficulty (that we discover through analyses in Section 4) are what informs both the “refinement”, i.e. how we divide the forget sets into subsets (using memorization levels), as well as how to pick the most suitable unlearning algorithm per subset. Now, RUM (as presented in the paper) makes the design decision of operating sequentially on refined subsets, which introduces this additional variable of the ordering, as the reviewer points out. Our work shows **the existence of orderings** that can yield large gains in unlearning pipelines. We hope future work extends our work to yield even larger gains in RUM, including a thorough investigation of the data ordering which is a very interesting component (but well beyond the scope of this work). **Overall and proposed adjustments to the paper:** We sincerely wish to thank Reviewer 3Cxb: (i) for engaging fruitfully and thoroughly with us and (ii) for their insightful comments that have already improved our paper. We are happy that 3Cxb appreciates the results with ToW-MIA, the example disagreement experiments, adding TinyImageNet and VGG-16 and acknowledges that, while unlearning definitions and subproblems may differ, the variant we study here is widely recognized as valuable. Taking reviewer 3Cxb’s comments to heart, we propose the following: When introducing ToW, we will clarify our motivation behind this metric (studying difficulty via a reasonable and informative scalar value), but we will also discuss variations of it in two axes: (i) per-example disagreement rate (unlearning vs retraining) in addition to average accuracy, and (ii) defining ToW-MIA (using MIA gap for forgetting quality, instead of forget accuracies). We will add sections in the Appendix to each of those points, showing the results of additional investigation in response to the reviewer’s feedback. We note that all of these variants show the same trends, which we will also note in the main paper, and we believe supports the robustness of our conclusions further. We thank the reviewer for bringing up these points which (we had not thought of and) strengthen our work. Please let us know if you would require anything else? Again, thanks!
Dear Authors, Sorry for the late reply, I was pretty occupied for the last few days. I have carefully read your rebuttal and additional experiments. And here are my further comments: - [**Leave-one-out**]: Leave-k-out makes sense and the new C-proxy experiments look great to me. You mentioned there is an order of magnitude improvement over Feldman's method in terms of efficiency. Still, I am curious about the comparison between the running time of the two algorithms. Can you provide a rough estimate on the CIFAR-10/ResNet experiment? - [**What makes unlearning hard**]: I like this clarification but I still think the current title could be a bit over-simplified. My suggestion is something like "What properties make data hard to unlearn". This is not critical though and I can also accept the paper with its current title. In summary, all the other concerns are well addressed. While I still think the contribution of "what makes unlearning hard" is more significant than "what to do about it", I am willing to recommend acceptance. I will adjust my ratings after another round of discussion with the authors. 7Yis
Addressing reviewer 7Yis new questions
Dear reviewer 7Yis, thanks a lot for your meaningful engagement. Please see below for answers to your questions: **Re: Leave-one-out... and “Can you provide a rough estimate on the CIFAR-10/ResNet experiment?”** Indeed, with C-proxy, running time is not an issue. Specifically, the running times of the two algorithms are as follows: - For computing Feldman’s memorization scores, we had to train 450 models! Training one model takes ca. 15 minutes. We used 4 GPUs, with the total time taking ca. **1,500 minutes**. - Computing C-proxy, on the other hand (which is done during the training of the original model) adds ca. 2.5 seconds per epoch, for a total of ca. **one minute of overhead**! This is a drastic difference in running times, showing negligible overhead for computing C-proxy, and negligible resource requirements. These facts allow us to claim that our contributions are practicable! **Re: What makes unlearning hard: I like this clarification … and … about the title**. We hope the above explanations help alleviate the original concerns about the practicability of our contributions (and the resulting concerns about the title). Again, thanks a lot for your positive feedback to our rebuttal. Please let us know if you have any other questions.
Final communication
Dear reviewer 7Yis, As the deadline is approaching, please let us know if there are any other concerns. Thank you.
Thanks for the detailed reply, I have read the rebuttal. My major concern about the efficiency is resolved by the new C-proxy method. I would maintain the current positive rating.
Thank you for engaging with us. We are glad to see that we have addressed your concerns regarding efficiency. We are very keen to engage further to resolve your other remaining concerns (regarding our claim that unlearning in LLMs is a substantially different problem with different definitions for the problem itself and for key concepts like memorization, and with different SOTA algorithms). Please do get in touch if there is anything else we can do to help alleviate any remaining concerns. Thanks again.
Response to rebuttal, other reviewers
Thank you for your thoughtful rebuttal. Your comments and results on efficiency address the majority of my concerns. Thank you for providing these details, I think they make for a stronger paper. I believe that this paper clears the bar for acceptance. I maintain my belief that a score of 7 is appropriate, since I think the work would make for a great poster. I have not assigned a higher score for this reason, but note that I am willing to fight for this paper's acceptance (if necessary, though it seems like it is not tending in that direction). I would to address some comments from other reviewers, which the authors have also addressed. I want to do so in support of their rebuttal. Forgive me if this is, as a result, a bit repetitive. I want to show places where I agree and disagree. *Reviewer Bp7U* > Despite the significance of the insights about sample embeddings, they appear somewhat limited to classification models, which typically have an embedding space to group different samples. While this may be beyond the scope of this paper, it would be valuable to explore extending these investigations to generative models, such as diffusion models and transformer-based large language models (LLMs). Machine learning remains about much more than LLMs. While I think the authors have provided a good rebuttal about doing unlearning on classification as being more generally useful (and while I understand why they may have engaged in a rebuttal like this), I don't think this was necessary. Having great results on classification is a contribution on its own. This paper doesn't need to touch LLMs to make useful contributions. > The experiments were conducted only on two CIFAR datasets, both involving image classification tasks. Including other tasks or datasets, such as text sentiment or topic classification, would make the paper more comprehensive. I want to support what the authors have said about this. Having done work in this area, it is tremendously expensive. (Related concerns can also be read in my own questions about efficiency -- this type of work is really expensive to do, so it's good to report on this. It also explains why experiments might be done on CIFAR, as opposed to other things). This probably should be listed as a limitation of the work, but is not in my opinion grounds for rejection. *Reviewer 3Cxb* I think that this reviewer raises important methodological questions about how machine unlearning success is measured in general in the field. In my opinion, the authors have done a good job of contextualizing challenges in the field in their rebuttal, in relation to the presented work. Such discussion is great for the field, and perhaps should go in the Appendix. Nevertheless, based on the comparisons and work done here, I still think that this paper pushes forward the state-of-the-art -- in a field for which there remains a lot of debate about how to measure success for unlearning. That is, while Reviewer 3Cxb raises some excellent points, I do think some of them read as critiques of uncertainty in the field of unlearning, and apply more generally than they do for this specific paper. This paper, in the allotted 9 pages, does a lot to try to improve rigor on measuring unlearning. It is, in my opinion, serving to provide more rigor in this still relatively unclear space. This in my mind serves to improve some of the challenges of unlearning, as articulated by this reviewer. Per-example forgetting results would be interesting and useful, but in my mind are not a blocker for publication of the existing work. *Reviewer 7Yis* I agree with the assessment that this paper ultimately helps clarify what makes unlearning hard perhaps more than what to do about it. I similarly think this is a sufficient contribution for publication as a poster.
Adding onto between-reviewer discussion
Thanks HxTF for tying our responses together. Quick note: >Per-example forgetting results would be interesting and useful, but in my mind are not a blocker for publication of the existing work. I agree with your assessment of my original review, and following the discussion with the author, I am convinced that the paper does make a reasonable contribution on the methods front (probably improving MIA by 3-5 points following their analysis + new method). My biggest concern remains with the example-level comparison. I basically think that aggregating accuracy across datapoints before comparing between models is clearly flawed, since the ToW metric could look good for models that produce very different distributions in practice (and I think pairs of neural models are often like this in practice). The authors convince me that this doesn't totally undermine their contribution, and I raised my score correspondingly, but I would strongly recommend to sort out this issue with the metric or otherwise provide substantial discussion and results in the appendix if they choose to keep the original formulation in the main body of the paper.
I will raise my score if this becomes necessary as a matter of record to advocate for accepting this paper. I hope this is not the case in discussion with the AC, since I think a 7 is fair (I personally would save 8 for the very small % of papers that warrant spotlights/orals).
Decision
Accept (poster)