Summary
Authors propose a method for identifying training images that need to be removed from the training set of a generative model to prevent a single specific “bad” (undesired) output from occurring in its output. Authors propose to directly unlearn the synthetized “bad” image, evaluate how this unlearning changes the training loss across training examples, and retrain the model omitting highly effected training examples - instead of evaluating how including each training example affects the “bad” output. Authors found that the resulting approach worked best when combined with prior techniques for preventing catastrophic forgetting in classifier uncleaning (e.g. optimizing only a set of weights and performing unlearning steps using Newton-Raphson iterations on the unlearned example using the original Fisher matrix). Authors show that retraining generators on pruned training sets indeed prevents the retrained generator from generating the undesired image - more specifically, authors show that the loss on the undesired output increases (more so compared to baseline methods) and that rerunning the retrained generator with the same random seed and text conditioning does not produce the “bad” example indeed.
Strengths
While at first the result might seem trivial (since authors both optimize and measure the same metric - “the model loss”) the finding is actually in fact very much non-trivial since authors retrain the model while pruning images that were affected most by unlearning the problematic output, so it might have so happened that removing these images did not prevent the output from occurring - so reported findings are indeed both interesting and valuable for the community (potentially beyond unlearning work).
Authors provide a random baseline and all provided results are significantly different from random (confidence intervals seem unnecessary).
Authors provide qualitative evaluation (Fig 3) suggesting that the desired effect is indeed observed in practice and not only in metrics.
Overall, the paper is well written and does a very good job of motivating both the problem and the proposed solution. Authors provide an extensive literature review that manages to be helpful even to a person without prior experience with unlearning (me).
Weaknesses
While I enjoyed reading the first half of the paper, I have serious concerns regarding the quality of the results section. While authors provide many qualitative results in the main paper, the burden of making sense of quantitative results (e.g. comparing them to ablations) is not only entirely on the reader, but is made worse by complete lack of any raw numbers for cross-comparison (e.g. no tables - only barplots w\o numbers), and putting the vast majority of quantitative results into supplementary without providing that much more context there. For example, Figure 4 (top) - colors are very similar and lines are very close, making results not really legible. Given that performance is evaluated only at three data points, I see no reason why this can not be a table - that would aid both reproducibility and comprehension (e.g. cross-comparison). I also do not quite understand the point of reporting the "equivalent to X random points” metric - it is never even explicitly discussed in the main paper. All results for MSE and CLIP metric and all Ablations (also no tables, only bar plots without numbers) are in the supplementary, but even there not much additional context for interpreting these results is provided.
Some claims are discussed and are just never qualitatively verified - e.g. authors claim that the proposed technique is specifically designed to not lead to catastrophic forgetting, but I could not find any evidence that would support that. Moreover, from Figure 3 it might seem like it not only forgot how to generate this specific bus, but might have forgotten how to generate all buses - I am not entirely sure if that is the intended behavior. I would appreciate a qualitative confirmation that the model did not just “forget everything”.
I have minor concerns regarding the delta G metric. While Georgiev et al. [12] supposedly already showed in an accepted (workshop) submission that two models trained independently on the same or similar datasets generate similar images when primed with the same noise input - justifying the existence of metric (3) - I’d appreciate some results confirming that this is indeed the case for these models in these experiments. For example, if authors showed at least qualitatively that removing images most influential for generating a particular “bad” bedroom from the training set does not indeed in any way affect the generation of unrelated images (e.g. an image of mountains), that would help. Or, even better, by performing textual inversion on the “undesired” input after removing images that affect it the most, to show that it can no longer be represented by the model. Otherwise, there is no way to tell if a particular “bad generation” just “migrated” elsewhere.
Another minor concern: since authors performed multiple gradient update steps, I wonder whether other methods can also be used with multiple update steps.
Questions
In addition to the weaknesses listed above, here are some questions/concerns I had while reading the paper:
Figure 1 - it is not said explicitly, but after several attempts, I figured out that the “bad” output that we want to remove in this figure is “bedrooms”, so it makes sense that the method assigned high loss (bottom orange bars) and “picked” bedroom images, the fact that only a single image (mountain) either “picked by the method” or “remains in the training set” is confusing; it might be worth to keep only two bedrooms and add a couple more images with low loss and somehow highlighting that “bedrooms are picked for removal”, not “only a single mountain is picked for keeping”.
L130 vs L104 - it is not clear why you need to introduce \tilda \theta if you already have \theta, especially given that in (2) RHS appears to not depend on \theta (w\o \tilda) and L157 also uses \theta (w\o \tilda), so the use of \tilda there is somewhat inconsistent and confusing; looking at L483-499 I think I understand now that the implied difference between the two is “optimal” vs “updated”, but i think this distinction is somewhat confusing in the main paper
L132 - \epsilon is never introduced at this point, so this is a bit confusing
L137-139 - trained from the same random noise or generated from the same random noise? worth rewording
L168-172 - these lines appear to not follow from the previous paragraphs introducing the notation for the attribution algorithm since it already provides per-sample scores (and therefore does not require 2^K evaluations); or not, and each tau(\hat z, z_i) is assumed to potentially involve iterating over all subsets of the training set (w\ and w\o z_i) - if so, please elaborate
Figure 2 - “Notably, our method better matches the poses of the buses (considering random flips during training) and the poses and enumeration of skiers.” - I would not say that it is very apparent from the figure.
Limitations
Authors do mention some limitations of their work.