Localizing Memorization in SSL Vision Encoders

Summary

This work focuses on memorization in self-supervised learning. This paper seems to extent the work from [1] on SSLMem, that leverage the fact that a given training data and its corresponding handcrafted image augmentations should have a lower distance in the SSL embedding space than with a model in which this point was not in the training data. The authors extend SSLMem by using intermediate layers instead of the final representation. By doing so, the authors can localize which layers are the most memorizing training data. They show that indeed the last layer memorize the most, however we can still find significant memorization occurring also in intermediate layers. The author improve then their method by introducing a memorization metric based on individual units instead of a layer basis. [1] Memorization in self-supervised learning improves downstream generalization, Wang et al, ICLR 2024

Strengths

The paper is very well written and the authors provide extensive experiments. The author provide results on different architecture, SSL methods, training criteria and augmentations. The results are strong.

Weaknesses

- The paper can be read as just an extension of [1], so this contribution could be seen as lacking novelty. However, the empirical analysis that is provided might still interest the research community. - I would be careful with claim such as being "first metric for localizing memorization". When we read Meehan et al., we can see that they did an ablation study across different layers, so they seem to be able to analyze which layer memorize the most. So it seems that there should have been a closer analysis and comparison with this paper results. In addition, they also provide experiments with ViT, so the author might also downplay the claim that investigating memorization in ViT is lacking. So one weakness of this paper is the lack of empirical comparison with respect to the current literature. - Even if the authors did a great job in trying different models, they did not analyze on a sample level basis, how the memorized examples might differ depending on random seed or different hyper-parameters. Like for two models that produces similar LayerMem score, are they memorizing the same examples/data points or not? Because both the overall score could be similar while memorizing different examples. A similar question is, does the memorize examples change across different layers? I would have appreciated reading at least some discussion on the relationship between model memorization with respect to which examples and how much those examples are consistent between models and layers. - Concerning the SSLMem metric, I am wondering what is the interplay between both models (training with and without specific points) initialization and hyper-parameters. Like I would suspect that depending on the data shuffling seed, the examples that might be memorized first in the early layers might differ. - There is also a lock of discussion on the impact of normalization layer or weight decay on the experimental results. I would suspect that having a stronger weight decay might reduce memorization.

Questions

- When replacing random or most/less memorized layers, how do you deal with normalization layers like batch norm since the statistics might be off? - How much time does it take to train both models, compute augmentations and get the LayerMem and UnitMem metric?

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors provide a limited limitation section. I would have appreciated to see as limitation that the method require training two disjoint model while having access to the original training data augmentation set. Thus, it does not seems that such method might work well on public model for which we do not know well the training details. Also the lack of a sample level analysis of the memorized example is an important limitation.

Summary

This paper introduces two novel metrics, LayerMem and UnitMem, to measure where memorization occurs within self-supervised neural networks. Through extensive experiments, this paper finds that memorization increases with layer depth, highly memorizing units are distributed throughout the encoder, atypical data points cause higher memorization, and most memorization happens in fully connected layers in ViTs. These findings suggest that localizing memorization can enhance fine-tuning and inform pruning strategies for better model performance and efficiency.

Strengths

- This paper proposes the first metric to measure memorization in a self-supervised learning context. - It presents several interesting and novel findings. For example, it's insightful that the fully connected layers in ViTs memorize the dataset more than the self-attention layers, like in NLP tasks. Also, it is interesting to see that memorization does not significantly depend on the self-supervised learning methods employed. - Overall, the paper is well-written and logically organized.

Weaknesses

- The primary limitations of this paper are the lack of in-depth analysis and practical insights. Although the paper provides a broad range of empirical findings, it could be significantly improved by exploring how and why the neural networks behave as they do. Alternatively, strongly linking to or elaborating on prior work might be a promising direction to enhance the paper. Moreover, it would be beneficial if the paper provided practical methods to improve self-supervised neural networks based on the given takeaways. Even though Section 4.2 was interesting and insightful, it is insufficient for applying these findings in a real-world setting. - If I understand correctly, I am not fully convinced by the definitions of `LayerMem` and `UnitMem` as appropriate metrics to measure memorization. For example, according to Equation 1, LayerMem seems to depend on the neural network’s properties. If neural networks are sufficiently equivariant with respect to data augmentation, then SSL and SSLMem should be zero. However, fortunately, the equivariance of ViTs is on par with that of CNNs [1], so I believe the main takeaways can hold in this case. Also, these metrics might depend on the norms of intermediate representations. - There is room for improvement in the writing. The paper claims that “memorization increases but not monotonically,” yet I observed that LayerMem increases monotonically and even uniformly, except in some layers, as shown in Tables 1, 14, 15, and 16. Some interesting results, e.g., Lines 247 and 387, are only included in the Appendix. The paper attempts to correlate atypical datasets and memorization, but the evidence does not strongly support the claims. It is a minor thing, but I’m not familiar with ‘SL’ as an abbreviation for supervised learning. Overall, considering the paper as one of the first attempts to investigate memorization in a self-supervised learning setting, I don’t find major weaknesses. Despite some limitations, I lean toward acceptance since I believe such attempts should be encouraged. [1] Gruver, Nate, et al. "The lie derivative for measuring learned equivariance." arXiv preprint arXiv:2210.02984 (2022).

Questions

- In Figure 2, why does SSL UnitMem start with a non-zero value even at the first layer? I expected it would start near zero. Also, the values appear to be almost the same across all layers. - Although high LayerMem values are observed in the later layers, this might be insufficient to conclude that these layers memorize more than the early layers. I anticipated that the effect of memorization would accumulate as the network depth increases. - Please refer to the weaknesses section for additional comments.

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

Please refer to the weaknesses section. No ethical concerns found.

Summary

This paper identifies where memorization occurs in SSLs, noting that memorization increases in deeper network layers, though high-memorizing units are distributed throughout the network. For the first time, they introduce a metric to measure memorization in SSLs and provide justification for its validity. They demonstrate that memorization primarily occurs in the fully-connected layers of vision transformers.

Strengths

* This paper proposes a metric to define memorization in self-supervised models and identifies the locations where the most memorization occurs.

Weaknesses

* It is not clear why the defined metric is a good measure of memorization, especially for comparing memorization across different layers. The magnitude of the distance between representations of different augmentations is highly dependent on the activation distribution of the layers and the distance metric used. While the normalization proposed in equations 2 and 4 attempts to address this, I am still unsure if it is sufficient. * To justify the validity of the LayerMem memorization metric, they argue that effective memorization in SSLs leads to strong downstream performance. They also point out that replacing the layers with the highest memorization scores, as determined by the metric, results in a more performance drop, indicating the metric's reliability. However, I believe this justification alone is insufficient. Are there other pieces of evidence supporting the validity of this memorization metric?

Questions

* When replacing the two layers of models trained on CIFAR-10 and STL-10, if the replaced layer is at the beginning of the network, don't the activations propagate through the entire network? I don't think I fully understood that experiment. * Is L2 distance used as the distance metric? If so, what is the justification for using it, and do the results change if a different metric is used?

Rating

5

Confidence

4

Soundness

2

Presentation

3

Contribution

2

Limitations

Yes

Summary

This paper investigates memorization in self-supervised learning encoders. It introduces two metrics, LayerMem and UnitMem, to locate memorization on a layer-wise and unit-wise basis. These metrics provide insights into the distribution of memorization within neural networks. The study reveals that memorization occurs throughout the layers of SSL encoders, not just in the final layers, and that, in vision transformers, memorization primarily takes place in fully connected layers. Furthermore, the authors propose practical applications for their metrics, such as enhancing the efficiency of fine-tuning and pruning through memorization localization.

Strengths

- The empirical evaluations follow a logical flow, utilizing state-of-the-art architectures and datasets, resulting in informative results. Overall the empirical results are quite extensive and thorough. - The proposed metrics are computationally efficient and practically convenient, requiring only a forward pass and no labels. Additionally, the UnitMem metric offers an intuitive and insightful definition. - The paper is overall very well-written and the topic/motivation is interesting.

Weaknesses

- There are some missing explanations on some of the observations. In addition to presenting empirical results, the authors should include explanations and insightful remarks to clarify their intuition and the potential implications of these results. This is done to some extent already, but would improve the paper if it is done in all the sections. - The presentation of the results could be improved. Tables full of numbers with tiny differences after the decimal point make it hard to read and navigate over. Instead of using tables that explain themselves, the paper relies on text explanations. - The proposed metrics, while inspired by previously known metrics, lack complete novelty. The LayerMem metric is primarily based on the existing SSLMem metric, with the only difference being a summation operation. This incremental modification does not constitute significant innovation.

Questions

- I am puzzled on why the memorization pattern remains the same between different datasets. For example in Figure 1 the histogram of UnitMem remains unchanged regardless of variations in the dataset or augmentations. Why does this happen? Do the authors have some suggested explanations for this? - Besides using these metrics in fine tuning are there other practical benefits to these proposed metrics?

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors discuss their metrics limitations to some extent.

Have concerns been addressed?

We would like to thank the Reviewer for recognizing our work as interesting, practical, and well executed. The paper has definitely improved as a result of the feedback. In summary, our rebuttal addressed the following: 1. **Additional Insights**: We improved the writing by adding additional insights, observations, explanations, and insightful remarks. 2. **Augmentations**: We performed additional requested experiments that show that stronger training augmentations lead to higher UnitMem. 3. **Practical Application**: We provided a list of practical applications for our work, where apart from improved fine-tuning, it could also be used for the memorization-based compressing schemes, strengthening the membership inference attacks, and watermarking. We would like to check if there are any pending concerns that would further strengthen the reviewer’s confidence for acceptance of our paper?

Have concerns been addressed?

We would like to thank the Reviewer for recognizing our work to proposes the first metrics for localization of memorization in SSL. In summary, our rebuttal addressed the following: 1. **Validity of the measure**: We present additional conceptual and experimental results validating that our metrics localize memorization. 2. **Clarifications**: We provided clarification on our experimental insights. 3. **Distance metric**: We performed additional experiments highlighting that our LayerMem metric is independent of the underlying distance metric. We would like to check: are there any other pending concerns that we should address?

Have concerns been addressed?

We would like to thank the Reviewer for recognizing our work as the first to propose metrics for localization of memorization in SSL encoders and presenting insightful as well as interesting findings. In summary, our rebuttal addressed the following: 1. **In depth analysis**: We extended the analysis of the paper and included the study on which samples are memorized in which layers. As requested, we added more explanations to the paper (using the additional 1 page if accepted). 2. **Validating the metrics**: We provide additional experimental and conceptual insights to validate our metrics, in particular, we show that the design of LayerMem is independent of the norms in the intermediate representations. We would like to check: are there any pending concerns that we should address to further strengthen the reviewer’s confidence for acceptance of our paper?

Have concerns been addressed?

We would like to thank the Reviewer for recognizing that our work enables localizing memorization, down to the granularity of individual units. In summary, our rebuttal addressed the following: 1. **Adjusting the claims**: We adjusted the claims according to the reviewer’s suggestion to recognize Meehan et al.’s contributions on analyzing ViTs and performing the ablation study on which layers hold highest accumulated Deja Vu memorization. 2. **Comparing memorization between layers**: Based on the reviewer’s suggestion, we performed additional experiments to study how memorized samples differ between the layers of a single encoder and between the layers of two different encoders. We show that especially adjacent layers memorize similar data points, and that over different encoders, the overlap of most memorized samples is most similar in later layers. 3. **Weights decay**: We performed also additional experiments based on the reviewer’s suggestion to show how UnitMem changes under different strength of weight decay and found that memorization decreases as weight decay increases, yet, also downstream performance drops. We would like to check if there are any pending concerns that should address to further strengthen the reviewer’s confidence for acceptance of our paper?

Higher memorization with stronger data augmentation

I would like to thank the authors for their response and running additional experiments. In particular for the data augmentation experiment, I am puzzled by the results. The results show that the stronger the data augmentation during training, the higher memorization happens in a layer level. Why is this the case? Shouldn't it be the opposite? Why the units/layers experience more memorization if during training they have seen more data samples? Data augmentation would in principle decrease memorization. (btw, I think there is a typo in your response? I only see the results in the attached PDF for LayerMem but in the response it is written the results are for UnitMem?)

Thank you!

We thank the Reviewer for the response and for increasing the score. We are glad that our answers addressed the concerns. Since the rating is still on the borderline, we are more than happy to provide any additional insights if needed.