Response to Reviewer 3ixD [2/2]
> Q2. Since the dispersed convolution is a dense operation with a larger receptive field, it introduces additional computational costs. How significant is the computational overhead of the method?
Please refer to our response to reviewer URVR Q2. Thank you!
> Q3. Is the re-dilation described in Sec. 3.2 actually used in the method?
Yes, the re-dilation is used in all the settings. However, when the resolution exceeds 1280x1280, we need to further incorporate the noise-damped classifier-free guidance (ndcfg), and the dispersed convolution. We have documented all these details in the hyperparameters section of the supplementary. We also clarify this in the revision of the main paper.
> Q4. Is it possible to find a closed-form solution for the optimization problem in Eq. 4?
**There is a closed-form solution for dispersed convolution. One only needs to solve a least square problem and no training is needed.** The detailed method is as follows.
We follow the notation in our main paper. Given a convolution layer with kernel $\boldsymbol{k} \in \mathbb{R}^{r \times r}$ and a target kernel size $r'$. We find a dispersion transform $\mathbf{R} \in \mathbb{R}^{r'^2 \times r^2}$ to get a dispersed kernel $\boldsymbol{k}'$. Consider an input feature map $\boldsymbol{h} \in \mathbb{R}^{n \times n}$, we ignore the bias in convolution, since it will not influence our results.
Structure-level calibration and pixel-level calibration can form an equation set
$$\left\\{\begin{matrix} interp_d (f_k(\boldsymbol{h})) = f_{\boldsymbol{k}'}(interp_d(\boldsymbol{h})) \\\ \eta f_{\boldsymbol{k}} (\boldsymbol{h}) = \eta f_{\boldsymbol{k}'}(\boldsymbol{h}). \end{matrix}\right.
\tag{1}$$
$ interp_d (f_k(\boldsymbol{h})) \in \mathbb{R}^{nd \times nd}, f_{\boldsymbol{k}} (\boldsymbol{h}) \in \mathbb{R}^{n \times n} $. The equation set has $(nd)^2 + n^2$ equations. Each equation is made up of the sum of terms $k_{ij}h_{ij}$ (elements in $\boldsymbol{k}$ and $\boldsymbol{h}$) where the coefficient of the terms are linear combinations of $R_{ij}$ (elements in $\mathbf{R}$) and constants. For example, when $n=3, r=3$ and $r'=5$, the 5-$th$ equation in pixel-level calibration is
$$
\begin{matrix} k_{11} h_{11} + k_{12} h_{12} + k_{13} h_{13} + \cdots + k_{33} h_{33} = \\\ (R_{7,1} k_{11} + R_{7,2} k_{12} + R_{7,3} k_{13} + \cdots + R_{7,9} k_{33})h_{11} + \\\ (R_{8,1} k_{11} + R_{8,2} k_{12} + R_{8,3} k_{13} + \cdots + R_{8,9} k_{33})h_{12} + \\\ \cdots + \\\ (R_{19,1} k_{11} + R_{19,2} k_{12} + R_{19,3} k_{13} + \cdots + R_{19,9} k_{33})h_{33} \end{matrix}
\tag{2}$$
We rearrange the equation and get
$$\begin{align}
(1 - R_{7,1}) k_{11} h_{11} + (-R_{7,2})k_{12} h_{11} + \cdots (1 - R_{19, 9})k_{33}h_{33} = 0 \notag.
\end{align} \tag{3} $$
To ensure this equation for all $k_{ij}h_{ij}$, one can let every coefficient be zero. Getting a linear equation set for the 5-$th$ equation in pixel-level calibration:
$$ \left\\{\begin{matrix} R_{7, 1} = -1 \\\ R_{7, 2} = 0 \\\ \cdots \\\ R_{19, 9} = -1
\end{matrix} \right. \tag{4}$$
We do this for every equation in Eqn.(1) to derive a larger set of linear equations of $R_{ij}$. Note that the equation sets are linear equations of $R_{ij}$ and have no $\boldsymbol{h}$ or $\boldsymbol{k}$, making it applicable to all conv kernels and any input feature. Lets back to general case where $\mathbf{R} \in \mathbb{R}^{r'^2 \times r^2}$. Let $A_{\mathrm{structure}}$ denote the coefficient of linear combination for $R_{ij}$ in structure-level calibration and let $b_{\mathrm{structure}}$ denote the right-hand-side constants in the equation set of structure-level calibration. Similarly, we define $A_{\mathrm{pixel}}, b_{\mathrm{pixel}}$. We construct a least square problem
$$A\mathbf{x} = b, \quad A = \left[\begin{matrix}A_{\mathrm{structure}} \\\ \eta A_{\mathrm{pixel}}\end{matrix} \right], b = \left[\begin{matrix}b_{\mathrm{structure}} \\\ \eta b_{\mathrm{pixel}}\end{matrix} \right], \mathbf{x} = \left[\begin{matrix} R_{1, 1} \\\ R_{1, 2} \\\ \cdots \\\ R_{r'^2, r^2}\end{matrix}\right]. \tag{5}$$
The solution is $\mathbf{x} = (A^T A)^{-1} A^T b$. This can be easily solved by math software, i.e., MATLAB. We have included this part in the supplementary materials.