Official Comment by Authors Part I
Thank you for your detailed review. We are pleased to provide answers to your questions:
## W1 ...theoretical insights will be strengthen their claim and more naturally lead to their MAP hypothesis...
Currently, we have not figured out how to directly derive MAP from the original DPS (Chung et al., 2022a). To compensate for this, we additionally provide a toy example, and derive MAP from a variant of DPS.
To better connect the original DPS and MAP, we provide a toy example below:
* The source distribution $p(X_0)$ is 2D 2-GMM (Gaussian Mixture Model). The centers are $(-1,-1),(+1,+1)$ with the diagonal $\sigma_0=0.3$.
* The operator $f(.)$ is inpainting. More specifically, the second dimension of the random variable is set to 0. And the measurement $y=(0.5,0)$
* The diffusion process uses 100 steps of Karras diffusion with $\sigma_{max} = 4$ and $\rho=7$.
* The diffusion scheduler is Euler Ancestral, and the original DPS $\zeta_t=0.05$.
* The source distribution, true posterior and the original DPS result is shown in this anonymous link: https://ibb.co/T2VSn32
* It is shown that the true posterior is a bimodal distribution. However, the samples from the original DPS cover only one mode of the true posterior. In this example, the behavior of the original DPS is closer to that of MAP.
On the other hand, with additional assumptions, it is possible to derive MAP directly from a variant of DPS: DSG (Yang et al., 2024). More specifically, DSG improves DPS with a spherical projection. The update of DSG is as follows:
$$
X_{t-1} = \mathbb{E}[X_{t-1}|X_t] - \sqrt{n}\sigma_t\frac{\nabla_{X_t}||f(\mathbb{E}[X_0|X_t]) - y||}{||\nabla_{X_t}||f(\mathbb{E}[X_0|X_t]) - y||||},
$$
__Assumption 5.__
Additionally, we need some assumptions:
* 1.The posterior mean $\mathbb{E}[X_0|X_t]$ is a perfect approximation to posterior samples from $p(X_0|X_t)$. In that case, the approximation error in Eq. (7)(8) diminishes, i.e.,
$$
p_{\theta}(y|X_t) = p(y|X_0=\mathbb{E}[X_0|X_t]) = \exp{-\zeta_t||f(\mathbb{E}[X_0|X_t]) - y||}.
$$
* 2.The $\sigma_t^2$ in Eq. (5) is so small that $\log p(y|X_{t-1})$ is locally linear in range of $\sqrt{n}\sigma_t$. The $X_t, X_{t-1}$ is so close that $\nabla_{X_{t-1}}\log p(y|X_{t-1}) \approx \nabla_{X_t}\log p(y|X_{t})$.
Under those assumptions, we can show that DSG is also a MAP:
__Proposition 6.__ The DSG solves the MAP problem $X_{t-1} \leftarrow \arg\max p_{\theta}(X_{t-1}|X_t,y)$ in __Hypothesis 2__ when dimension $n$ is high.
* __proof__: We first consider a gradient ascent with learning rate $\alpha_t$:
$$
X_{t-1} = \mathbb{E}[X_{t-1}|X_t] + \alpha_t \nabla_{X_t}\log p(y|X_t).
$$
As we have assumed in __Assumption 5.2__, the $\log p(y|X_t)$ is locally linear and $\nabla_{X_{t-1}}\log p(y|X_{t-1}) \approx \nabla_{X_t}\log p(y|X_{t})$. As gradient is the direct of steepest descent, when $\alpha_t$ changes, the $X_{t-1} = \mathbb{E}[X_{t-1}|X_t] + \alpha_t \nabla_{X_t}\log p(y|X_t)$ is the maximizer of $\log p(y|X_{t-1})$, given the distance from the point $X_{t-1}$ to center $\mathbb{E}[X_{t-1}|X_t]$ is fixed.
On the other hand, the DSG can be written as the above gradient ascent
$$
X_{t-1} = \mathbb{E}[X_{t-1}|X_t] - \sqrt{n}\sigma_t\frac{\nabla_{X_t}||f(\mathbb{E}[X_0|X_t]) - y||}{||\nabla_{X_t}||f(\mathbb{E}[X_0|X_t]) - y||||} \\
\overset{a}{=} \mathbb{E}[X_{t-1}|X_t] + \frac{\sqrt{n}\sigma_t}{||\nabla_{X_t}f(\mathbb{E}[X_0|X_t]) - y||||}\nabla_{X_t}\log p(y|X_{t}).
$$
where (a) holds due to __Assumption 5.1__. We further notice that DSG is equivalent to the above gradient ascent. Further, from the update procedure of DSG, the distance of $X_{t-1}$ to $\mathbb{E}[X_0|X_t]$ is $\sqrt{n}\sigma_t$. Therefore, the result of DSG is the minimizer of $\nabla_{X_{t-1}}p(y|X_{t-1})$, on the sphere surface centered at $\mathbb{E}[X_{t-1}|X_t]$ with radius $\sqrt{n}\sigma_t$:
$$
X_{t-1} \leftarrow \arg\max \log p(y|X_{t-1}), X_{t-1} \in \mathcal{S}(\mathbb{E}[X_{t-1}|X_t],\sqrt{n}\sigma_t),
$$
where $\mathcal{S}(\mathbb{E}[X_{t-1}|X_t],\sqrt{n}\sigma_t)$ is the sphere surface centered at $\mathbb{E}[X_{t-1}|X_t]$ with radius $\sqrt{n}\sigma_t$. Besides, from (Yang et al., 2024), we know that as dimension $n\rightarrow \infty$, $X_{t-1}\sim p(X_{t-1}|X_t)$ is equivalent to $X_{t-1}\in \mathcal{S}(\mathbb{E}[X_{t-1}|X_t],\sqrt{n}\sigma_t)$. Then as dimension $n\rightarrow \infty$, DSG+DPS is eqvalient to:
$$
X_{t-1} \leftarrow \arg\max \log p(y|X_{t-1}), X_{t-1}~p(X_{t-1}|X_t),
$$
which is again equivalent to __Hypothesis 2__ by __Proposition 3__. $\square$