Déjà Vu Memorization in Vision-Language Models

Summary

This work studies training data memorisation in Vision-Language Models (VLM). The paper focusses on contrastive learning with OpenCLIP, using a private Shutterstock dataset and a filtered LAION-50M, and evaluation on ImageNet. The paper proposes a method and metrics to measure déjà vu memorization, and in addition explores mitigation strategies. The paper concludes by stating it has demonstrated the presence of memorization.

Strengths

This paper is well-written, explores an important issue, and proposes both new ways of measuring the issue and ways of mitigating it. Many experiments are run, addressing many of the questions this work raises.

Weaknesses

**W1**: The main weakness of this work is that it does not sufficiently distinguish memorisation from learning. On line 162 the paper states: "If no memorization occurs, models fA and fB should be interchangeable and hence this gap is zero.", and while one would expect that two models trained on sufficiently large data would eventually converge - it is unlikely that the models being different before this point is purely due to memorisation. Two models trained on the first 500 and the second 500 classes of ImageNet-1K will have significantly different performance on in-distribution versus out-of-distribution samples, yet, memorisation does not seem like the most likely explanation for this. Similarly, with two models trained on the same dataset - with the exception of a single image-text pair, there may be a difference in performance. This paper attributes this to the memorisation of that image-text pair, however, I'd argue this could also be explained by learning - depending on the size of the original datasets and the 'importance' of the held-out data point. For instance, if the held-out data point is the only image-text pair to contain the 'cat' concept then the loss may be very high for this data point, and as such, it influences the network weights disproportionally. As the dataset size grows it becomes more likely that most concepts in the test set will have been seen during training, and thus the importance of any individual data point decreases. While memorisation may also be at play here, it is difficult to disentangle it from learning without a proper discussion about what distinguishes the two and how this may be seen in the experiments. In particular, if one takes the view that learning is compression of data points the boundary to memorisation becomes very blurry. **W2**: A second, but lesser weakness, is that the mitigation experiments do not explicitly address multi-modal nature of VLM - in section 5.4 it is discussed that images are already augmented, and an additional text masking strategy is proposed to match it. Yet, based on this parallel it would then seem logical that data augmentation of images also prevents memorisation - which is not explored. On the other hand, given how strongly the proposed memorisation testing strategy depends on the text prompts, it makes sense that the measured metrics drop - but this does not exclude image memorisation within VLM, which may be unaffected by this augmentation approach. **W3**: Two minor points: 1) Figure 1 is rather challenging to decipher before having thoroughly studied the text, and afterwards, the added value is minimal. Consider removing/updating this figure. 2) Line 233 discusses 'the adversary' which is not clear within the context of the paper.

Questions

The main question concerns W1, which is, how this this work distinguish between memorisation and learning, and to what extent can the two be disentangled in interpreting the result? If this point can be addressed I would switch to a more positive score, as I do think the work is interesting, but as long as it leaves open this alternative explanation I will recommend a reject.

Rating

4

Confidence

4

Soundness

2

Presentation

4

Contribution

2

Limitations

Not applicable.

Summary

This paper investigates the issue of overfitting to pre-training data in Vision Language Models (VLMs) like CLIP. The authors conduct a comprehensive set of experiments focusing on text-to-image retrieval to evaluate this phenomenon. Their findings indicate that VLMs often memorize the data encountered during pre-training, which can impact the performance of downstream applications. However, the experiments also reveal that this problem diminishes as the scale of pre-training data increases. Overall I think this paper is important for the community since it tries to understand the limitation of popular multimodal foundational models like CLIP. It does have certain limitations but I think it’s a novel and thorough evaluation conducted at understanding CLIP through the lenses of pretraining data.

Strengths

1. The paper is well written and easy to follow. The metrics and methodology used is properly explained. 2. Vision Language Models (VLMs) like CLIP have become crucial for downstream applications, including multimodal chatbots and text-to-image generation systems. Evaluating these models for their limitations is essential to improve their foundational capabilities. The authors assess these models by examining their tendency to memorize training data, a well-documented issue in traditional machine learning models, providing a strong motivation for this study. 3. The experiments carried out make sense and the authors also evaluate few methods to mitigate the issue and present empirical results of each method tried.

Weaknesses

1. While the authors evaluate four mitigation strategies, none of them effectively address the identified problem. It would have been beneficial to see a strategy that not only mitigates the problem but also enhances the model's utility. 2. I believe that Figure 6 should be included in the main paper. Since the authors discuss it in detail, having it in the main paper would make it easier to follow and understand.

Questions

1. Recent work has shown that the pretraining data of OpenCLIP models suffers from long-tail issues, I would like to to know if the authors evaluated this problem in a semantic sense as well? For eg concepts that are rare in the pretraining data, does the model tend to memorise these concepts more? [1] Parashar et al. (2024). The Neglected Tails in Vision-Language Models. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*

Rating

7

Confidence

5

Soundness

3

Presentation

4

Contribution

4

Limitations

1. Although the authors conduct experiments to evaluate the impact of data scale, such as scaling from 10M to 50M image-caption pairs, they do not perform orthogonal experiments to assess the impact of model size. Evaluating larger models, such as the ViT-L/14, would have been a natural next step and could have provided valuable insights. 2. I disagree with the assertion that VLMs are typically pretrained on the scale of 10M image-caption pairs. For instance, the first CLIP model was pretrained on 400M image-caption pairs, and subsequent models have used even larger datasets. Although conducting experiments on such large datasets may be challenging, evaluating this problem at the more common pretraining data scale would have been more relevant, especially since the authors state that the problem diminishes as the data scale increases.

Summary

This paper explores the concept of training data memorization within vision-language models (VLMs). The authors introduce a method to measure the degree of memorization by analyzing the fraction of ground-truth objects in an image that can be predicted from its text description. The study reveals a significant level of memorization, which is evaluated at both sample and population levels. The findings indicate that OpenCLIP retains information about individual objects from the training images beyond what can be inferred from correlations or image captions. Additionally, the paper demonstrates that text randomization can reduce memorization with only a moderate impact on the model's performance in downstream tasks.

Strengths

- *Pioneering Approach*: The paper addresses a critical and complex issue by proposing a novel method to measure memorization in VLMs. This method, despite its imperfections, lays the groundwork for future research, offering a baseline for further refinement and extension. This work opens up new avenues for exploring memorization in multimodal settings, which is a non-trivial task.

Weaknesses

- *Lack of Interpretability*: The major weakness of this work is the lack of interpretability of the proposed method. The proposed method is complex and not straightforward in its measurement of memorization. The reliance on an external object detector introduces additional biases and imperfections, complicating the interpretation of the results. For instance, the meaning of a PRG score of 0.17 is unclear, and the authors should provide guidance on interpreting these metrics. - *Absence of Baseline Comparisons*: The paper would benefit from comparisons with simple baselines to contextualize the proposed method's effectiveness. Although identifying suitable baselines is challenging, their inclusion could strengthen the validity of the findings. - *Clarity Issues*: Some aspects of the paper are difficult to understand. Figure 1 contains too much information and lacks a clear structure, making it hard to follow. Additionally, the results section is laborious to read due to the excessive use of acronyms.

Questions

- I understand that memorisation is an issue with generative models as they can regurgitate training data during inference time. But why is memorization important in non-generative models? What are the potential risks or drawbacks? I see that it could negatively impact its predictions, but is there something else? - Would reproducing the experiments with multiple pairs of fA and fB trained with different random seeds yield more robust results, or do the authors believe this to be unnecessary? - This paper tackles a challenging and important problem in the field of vision-language models. While the proposed method has limitations, its novelty and potential for future research make it a valuable contribution. This is why I am giving it a borderline accept. The main areas for improvement are enhancing the interpretability of the method (or convincing me that it is interpretable), including baseline comparisons (if possible), and improving the clarity of the presentation. With these improvements, the paper could make a stronger case for acceptance. [edit: Based on the rebuttal and the other reviews, I decided to increase my rating to 'weak accept'.]

Rating

6

Confidence

3

Soundness

2

Presentation

2

Contribution

3

Limitations

Limitations are adressed in the paper.

Summary

The paper proposes a methodology to measure memorization in vision-language models (VLMs). These measurements are based on the fraction of ground-truth objects in an image that can be predicted from its text description. The authors also explore different mitigation strategies.

Strengths

- The methodology is novel and useful in evaluating if the model is overfitted to the training data. - The paper shows extensive evaluation on both population and sample-level memorization. The ablation studies on mitigation methods are comprehensive. - The paper is well-structured and clearly written, explaining the methodology and results effectively.

Weaknesses

See Questions section.

Questions

- Have you considered combining multiple mitigation approaches? For instance, setting both weight decay and text masking rate to 0.3 could potentially yield complementary benefits. - Have you conducted experiments to compare the performance of models trained with different mitigation approaches on other tasks, such as retrieval or compositional reasoning benchmarks?

Rating

6

Confidence

2

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors have clearly addressed limitations in their paper.