Further response to follow-up comments (1/2)
We are grateful to see the follow-up comments. We address the additional concerns below. Feel free to let us know if you would like us to clarify anything further.
> **A2: Discussions of related “concurrent” work**
**Description of GeNVS**
We agree that the Matterport3D dataset used by GeNVS is similar to RealEstate10K, and we will correct its description to “GeNVS mainly works on 360-degree object-centric scenes *or nearby viewpoint scene-level datasets*”. Nonetheless, it does not alter the previous conclusion as it still belongs to the two existing scenarios summarized in L28-L30 and Fig. I of the one-page PDF. The key contribution of MVSplat360 is that it focuses on an *unexplored* setting: feed-forward *360-degree* scene synthesis. The *majority of experiments are conducted on DL3DV* (rather than RealEstate10K), which significantly verifies MVSplat360’s effectiveness in handling 360-degree NVS from sparse views.
**Discussions with ReconFusion**
* We refer the reviewer to the project page of ReconFusion for the results from the diffusion model. In particular, as shown in the last video entitled “ReconFusion distils a consistent 3D model from inconsistent samples” on ReconFusion’s project page, views sampled purely from its diffusion module are far from consistent, showing obvious jittering from frame to frame. Our supplementary video shows that MVSplat360 renders multi-view novel views with much higher consistency.
* Note that those inconsistent demos provided by ReconFusion mainly consist of forward-facing or *constraint* orbital camera trajectories, while our consistent demo contains way more challenging rendering trajectories, including different types of *unconstrained* trajectories.
* It is necessary for ReconFusion to apply per-scene optimization in order to get satisfactory consistent novel views, while our MVSplat360 renders 3D consistent novel views in a feed-forward manner thanks to our effective design summarised in “Contributions of MVSplat360 (Method)” in the global response to all reviewers.
> **A3: Training objectives**
We will update this potentially ambiguous claim regarding "enhancing geometry" in the paper.
* It is correct that the reconstruction loss is the only source of gradients for the structural parameters of the Gaussians.
* Our initial intention is that joint training can help “enhance the backbone features”. Since the features and other Gaussian parameters are from different heads but share the same backbone, we assume that enhancing the backbone feature will lead to a better reconstruction module and, hence better reconstructed coarse geometry. To avoid unintentional overclaim, we will tone down L168 to “the SVD loss can further optimize the Gaussian features, further *enhancing the reconstruction backbone*”.
> **A4: SD vs SVD**
**Ablation of SD**
In the updated version, we will make it clearer that we denoise all novel views jointly. We will also add visual comparisons with the SD-based ablation model, which shows inconsistent novel views compared to our default design, as observed in our experiments.
**When novel views are far away from each other**
We did not observe obvious limitations regarding this issue. This is probably because although the viewpoints range from 180 to 360 degrees for each scene in DL3DV, it still belongs to one scene and might not contain significant “far away” novel views.
**Test time trajectories**
* In our experiments, the set of novel views is always a plausible trajectory. In particular, for quantitative evaluation, we use the camera trajectories captured by the initial video. For the video demo, we apply a Gaussian filter to the initial captured camera path to obtain 6DoF stabilization results.
* Each trajectory contains 56 frames, which are uniformly sampled from the initial video that contains around 300 frames.
**Cross-view attention vs. concatenation**
In typical single-view or multi-view diffusion-based NVS models, their input features come from the *source/observed viewpoints*. In this case, they are required to align/correct those source viewpoint features to match with the novel view cameras, hence it might be better to achieve via cross-view attention. In contrast, in MVSplat360, the features provided to the SVD are those rendered from *target/novel viewpoints*, containing coarse but correct geometry information for the target viewpoint, hence it is reasonable to achieve via concatenation.
It might be helpful to add *additional source/observed view* features to the SVD refinement module via cross-view attention, keeping the current concatenation for *target/novel views*. We will explore this strategy in our further experiments. Thanks for the insightful suggestion!
*(see next comments for more response, thanks.)*