Official Response to Reviewer PDfA
We greatly appreciate your valuable feedback. Therefore, we have incorporated several additional experiments into the paper to showcase the effectiveness of DisPose. These newly added experiments are highlighted in $\color{red}{red}$ within the revised version of the paper.
> **Q1: The detailed disentanglement operation.**
1. The motion field guidance and key point correspondences are not input by the user, rather we disentangle them from the user's input of the driving pose.
2. As shown in Figure 2(c)(d) of the manuscript, the detailed operation consists of extracting the motion fields and keypoint correspondences.
3. Section 4.1 introduces the disentanglement operations for sparse and dense motion fields. Specifically, we first estimate the skeleton pose by DWpose. Then, we compute the trajectory matrix (Ps) and apply Gaussian filtering to obtain the sparse motion field (Fs). Next, we predict the reference-based dense motion field (Fd) via conditional motion propagation (CMP).
4. Section 4.2 presents the detailed operation of keypoint feature extraction. Specifically, we first extract the DIFT features of the reference image using the pre-trained image diffusion model and then obtain the keypoint embedding according to the keypoint coordinates in the reference pose. We initialize the keypoint correspondence map (Fp) according to the trajectory coordinates of the driving pose.
> **Q2: Reasons for differences in the calculation of Pd for training and inference.**
There is a gap between the inference and the training optical flow.
1. During inference, we do not propose extracting the forward optical flow directly from the driving video, as it ignores the gap between the reference character and the driving video. As shown in Figure 13(a) and Figure 14(a) of Appendix D, directly using the forward optical flow as motion guidance is clearly inconsistent with the reference image.
2. When there is a large difference between the reference image and the driving video, it is impossible to get the corresponding motion field by the existing optical flow estimation model, as shown in Figure 14(b) of Appendix D. Therefore, we have to compute Pd differently during inference.
>**Q3: Reasons for sampling before prediction.**
Although the dense motion field we proposed in Section 4.1 can adapt to different body variations during inference. However, there are two limitations of this dense motion field: (1) there is a gap with the motion field extracted from real videos, and (2) the low computational efficiency is not suitable for use during training. Considering that pairs of training data have no body changes, to utilize accurate control signals during training and to improve computational efficiency, we approximate the optical flow during inference by sampling sparse optical flow before prediction, as shown in Figure 13(c) of Appendix D.
> **Q4: The effectiveness of multi-scale point correspondence.**
1. First of all, the key points of the human body correspond to the most distinctive appearance characteristics. We capture the appearance characteristics of the reference character by extracting the diffusion features corresponding to the keypoint coordinates in the reference image. Notably, our keypoint correspondence is not just a set of coordinates; it is a set of feature maps that represent key appearance information.
2. Moreover, Figure 7 in the manuscript shows that keypoint correspondence can localize to the correct semantic region during cross-pose and cross-identity, which justifies that keypoint correspondence can provide information about appearance content. Table 3 in the manuscript demonstrates that removing keypoint correspondences affects various metrics.
3. To move a step further, we provide qualitative experiments in Figure 11 of Appendix D, which prove that keypoint correspondence contributes to appearance consistency.
> **Q5: Description of the Point Encoder from Section 4.2.**
To seamlessly integrate into existing models, we introduce keypoint correspondences into the intermediate features of the hybrid control network encoder. Thanks to your suggestion, we have updated Section 4.2 of the manuscript.
>**Q6: More realistic generation results.**
As discussed in Section 8 of the manuscript, we provide the qualitative results of the synthetic characters to avoid leakage of personal privacy. Thanks for your thoughtful comments. We provide real-life results on the project page (https://anonymous.4open.science/r/DisPose-AB1D). Please do not distribute or divulge these personal portraits.