Further response for Reviewer crn6's feedback
Sincerely thank you for your feedback. We have some further responses w.r.t. (a) 4D reconstruction and (b) special designs for generated videos.
**(a) 4D reconstruction**
Since our focus is on the 4D reconstruction, to the best of our knowledge, there are no **pose-free** benchmarks for **dynamic scenes**. To address your concern, we build the benchmark by collecting openly available videos from the SORA official webpage. We then strictly compare our method against existing state-of-the-art 4D reconstruction methods. We provide the details and results below.
**Benchmark details:** We collect 35 sub-videos from the SORA [1] webpage, including Drone_Ancient_Rome (20 seconds in total, split into 4 sub-videos), Robot_Scene (20 seconds in total, split into 3 sub-videos), Seaside_Aerial_View (20 seconds in total, split into 3 sub-videos), Mountain_Horizontal_View (17 seconds in total, split into 3 sub-videos), Snow_Sakura (17 seconds in total, split into 4 sub-videos), Westworld (25 seconds in total, split into 4 sub-videos), Chrismas_Snowman (17 seconds in total, split into 3 sub-videos), Butterfly_Under_Sea (20 seconds in total, split into 3 sub-videos), Minecraft1 (20 seconds in total, split into 4 sub-videos), Minecraft2 (20 seconds in total, split into 4 sub-videos).
**Evaluation method:** We follow the standard pipeline for dynamic reconstruction (Hyper-NeRF, SC-GS, etc), to construct our evaluation setup by selecting every fourth frame as a training frame and designating the middle frame between each pair of training frames as a validation frame.
**Results:**
| Method | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ |
| :---: | :----: | :---: | :---: |
| Deformable-GS | 12.72 | 0.5773 | 0.2861 |
| 4D-GS | 12.15| 0.5609 | 0.2926 |
| SC-GS | 14.81 | 0.5914 | 0.2420 |
| SpacetimeGaussians | 13.24 | 0.5836 | 0.2633 |
| Ours without field initialization (Sec. 3.3) | 15.42 | 0.6167 | 0.2268 |
| Ours without dual branch refinement (Line 187) | 18.57 | 0.6852 | 0.1945 |
| **Ours (full model)** | **19.05** | **0.7323** | **0.1839** |
Upon acceptance, we will open-source this benchmark and the corresponding codebase to ensure its reproduction. Besides, since currently there are no pose-free benchmarks for dynamic scenes, we believe this built benchmark is a contribution.
Our full model achieves 4.24 PSNR improvement compared to the existing best method. The result proves the superiority of our method and each element we propose (field initialization, dual branch refinement) on the pose-free dynamic scene benchmark.
**(b) Special designs for generated videos**
As we summarized in the common response, properties of generated videos include both larger-scale aspects (unknown poses and unexpected movement) and small-scale aspects (flickering, floater occlusion).
- For larger-scale aspects, we propose the Field initialization stage which provides a proper start for our Dynamic Gaussian Surfels (DGS) regarding both the pose and the movement (please see warping transformation in Eq. 6 of our main paper). Here we'd like to highlight that the field initialization also benefits movement learning since the warping transformation is learned as a continuous field. This design is novel and especially beneficial for generated videos. We will provide more details of the field initialization during revision.
- For small-scale aspects, we have proposed the Dual Branch Refinement (Line 187) and provided ablation studies to prove its effectiveness in alleviating flickering.
Again we are grateful for your feedback.
[1] Video Generation Models as World Simulators.