Summary
Based on the manifold assumption, this paper analyzes the memorization phenomenon in diffusion models. Theoretically, the authors study the memorization effect on an empirical data distribution with its underlying data distribution as a Gaussian distribution. They show that in such case, the specific sampling of the training data points can introduce variances that lead to losing dimensions, i.e., some directions of the tangent space of the manifold are lost. Beyond this theory, they provide empirical results on both synthetic and real world dataset to support the theoretical results.
Weaknesses
The analyze in section 5 is itself very interesting and meaningful, however, I have questions on some of the definitions. Furthermore, I think there exists huge gap between the theoretical results and the practice. i list my questions as follows:
(i) First of all, I have doubt on the definition of the manifold (equation 3). Can you explain why the manifold is defined as the points that has 0 score? This is a very restricted definition since in practice, the $x_t$ along the reverse sampling trajectory never satisfy this assumption unless $t$ is extremely close to 0. Shouldn't we define the manifold as the intermediate noisy distribution $p_t(x)$ ? For the experiment results on natural images, can you clarify whether you are calculating the Jacobians at $x_t$ that have zero score ?
(ii) Without this special manifold assumption, the singular vectors with zero singular values no longer represents the tangent space. Can you clarify whether/how the tangent space interpretation holds when analyzing models trained on natural images ?
(iii) In contrast, the work by Stanczuk does not have this special manifold definition and the dimension of the manifold is calculated in a very different manner rather than just counting the number of zero singular values. I suggest that the authors compare their approach more directly to Stanczuk's method, explaining the key differences and potential implications.
(iv) The definition of memorization is very different from what people care about in practice. In practice, people define memorization as diffusion models generate images that can be found in its training dataset [1]. To be more specific, given a noisy image $x^*_t$ and score function $s(x^*_t,t)$, the denoised version of $x^*_t$ can be written as $x^*_t+\sqrt{t}s(x^*_t,t)$. and memorization happens when $x^*_t+\sqrt{t}s(x^*_t,t)$ is close to training images of the diffusion models. Can the author address how their definition of memorization relates to or differs from the practical definition I've described ?
(v) I think the Jacobians matrix alone cannot fully explain the memorization phenomenon since it doesn't have enough information on the score function $s(x^*_t,t)$. Notice that equation (4) only holds when the deep network is bias free. In general, equation (4) should be written as:
\begin{align}
s(x^*+p,t)\approx s(x^*,t)+J(x^*,t)p.
\end{align}
I think memorization depend more on the property of $s(x^*,t)$ itself, rather than its Jacobian matrix. Can the author explains how their choice of focusing on the bias-free deep network might limit the analysis?
(vi) The analysis on empirical score has huge gap between practice since recent work[2] has shown that denoisers in practice cannot learn such empirical score function. For this reason, the emprical score function is too ideal and cannot explain the empirical results on natural images (section 6.3).
(vii) The paper claims that memorization causes the model to losing some directions of the tangent spaces. They support this by observing that the time-dependcy gaps in the Jacobian spectrum are lost. However, it should be noted that for diffusion models trained on different number of images, the singular vectors of the Jacobians change significantly as well, which can be observed in the following link
\url{https://github.com/LabForComputationalVision/memorization_generalization_in_diffusion_models/blob/main/notebooks/Demo_UNet_CelebA80x80.ipynb}. Therefore, "losing dimension" is not an accurate description since the subspaces themselves change significantly. The current theory cannot explain this change in subspaces.
[1] Wang, Wenhao, Yifan Sun, Zongxin Yang, Zhengdong Hu, Zhentao Tan, and Yi Yang. "Replication in visual diffusion models: A survey and outlook." arXiv preprint arXiv:2408.00001 (2024).
[2] Zeno, Chen, Greg Ongie, Yaniv Blumenfeld, Nir Weinberger, and Daniel Soudry. "How do minimum-norm shallow denoisers look in function space?." Advances in Neural Information Processing Systems 36 (2024