Epipolar-Free 3D Gaussian Splatting for Generalizable Novel View Synthesis

Thank you for your thoughtful response. Please find below our detailed comments on the novelty of our work and an analysis of the performance gain. ## **Novelty of the Work:** * **Epipolar-Free Approach to Address Overlap Challenges.** We have identified that mainstream multi-view GNVS methods, which rely on epipolar geometry, may encounter significant challenges in scenarios with large viewpoint overlaps. To address this, we explored the role of cross-view pretraining models that provide 3D prior knowledge for the GNVS task, and we introduced a cross-view mutual perception mechanism to partially mitigate the limitations of current methods. * **Iterative Cross-view Gaussians Alignment Module.** Beyond cross-view mutual perceptive pretraining, we have specifically introduced the Iterative Cross-view Gaussians Alignment module. We observed that relying solely on the CroCo pretraining model is insufficient to effectively address challenges such as ambiguity in GNVS, primarily due to the absence of precise local feature matching. Unlike previous generalizable NVS methods utilizing Gaussian Splatting (e.g., MVSplat), our approach circumvents the need for time-consuming and less generalizable structured feature representations, bypasses complex depth sampling processes, and enables efficient per-pixel Gaussian attribute prediction. The ablation experiments presented in Table 3 and Figure 6 demonstrate the critical importance of the Iterative Cross-view Gaussians Alignment module. **Our approach not only effectively manages scenarios where epipolar geometry may fail but also eliminates the time-consuming process of sampling along epipolar lines. This makes our method particularly efficient and effective for GNVS tasks, especially in scenarios with extensive viewpoint overlaps.** ## **Performance Gain Analysis:** With regard to Table 2 of the manuscript, the three subsets with overlaps below 0.7, 0.6, and 0.5 each represent challenging experimental conditions where epipolar geometry becomes unreliable, and it would be inaccurate to assert that one is inherently more challenging than the others. **Moreover, the performance gain across these subsets does not follow a linear progression.** The reasons are as follows: * **Distribution Gap Between Subsets and Training Set.** These three low-overlap subsets differ from the training set and constitute a small fraction of the test set. The majority of training data involves input viewpoints that overlap well above 0.7. In the test scenes, the ratio of scenes with overlaps greater than 0.7 to those below 0.7, 0.6, and 0.5 is 45:8:5:1. Thus, from a dataset distribution perspective, these three subsets exhibit significant variability, and the overlap size does not directly correlate with difficulty. This distribution gap explains why the performance gain remains similar across these subsets, as the model’s performance is more heavily influenced by distributional differences than by the specific overlap percentage. * **Non-linear Error Growth with Decreasing Overlap.** In 3D computer vision, local matching errors can propagate and affect the global output, meaning that overall error does not necessarily increase linearly as overlap decreases. This is because incorrectly matched features might be propagated as outliers into subsequent modules (especially in modules with global awareness, such as transformers), thereby amplifying their impact on overall performance. Once overlap falls below a certain threshold, traditional methods are significantly compromised. For instance, if the feature point matching error rate is too high, SFM algorithms may fail. Consequently, scenes with overlaps below 0.7–0.5 all represent scenarios where epipolar geometry is unreliable, and there is no strictly linear relationship for such "unreliableness". Thus, the similarity in performance gains for overlaps of 0.5 and 0.7 reflects the model’s consistent capability in managing these challenging scenarios where epipolar constraints are inadequate. Finally, thank you again for your excellent suggestions, which have helped us to think more deeply about the essence of performance improvement in our method. We will incorporate this analysis of the performance gain into the revised manuscript.

Thanks for the detailed and helpful reply! The motivation of using epipolar-free methods makes sense to me. While this field is super competitive as there are many recent works, releasing unnecessary inductive bias on geometric designs is promising to me, especially when having large data. The competition in this field makes me a bit tired, as I can feel most of the works are a kind of incremental. I appreciate the effort to further improving Dust3r training pipeline on the GNVS task using the proposed epipolar-free method and the iterative cross-view gaussian alignment method. Regarding the performance analysis, maybe the reason for my previous question is the data distribution (45:8.5:1 as mentioned). The data amount with small overlap ratio, e.g. 0.5, can be too less, so the numbers are also noisy. I sincerely suggest the authors to find some other data that are more balanced to perform the analysis, e.g. DTU or DL3DV, where you can control the overlap ratio as the two datasets provide dense views. I won't request this experiment as the discussion period is coming to the end soon. At the same time, we still want to let the authors know I expect the prior works can be correctly discussed, especially removing some controversial words like first "the first epipolar-free GNVS method". After the discussion, I don't want to reject this paper because of the overclaiming problem as I saw the effort behind this paper. Thus, I would like to raise the score to **a neutral borderline** (with a score of 4.5, due to the uncertainty of how the over-claiming problem will be resolved) and I don't against accepting this paper. As there is no neutral borderline for the reviewer, I will just kept the current rating now, but I will let AC know if I keep the a neutral borderline until the end of discussion. **At the same time, if you have better wording for resolving the controversial claims, please let me know before the discussion period ends. If the revised version is satisfying to me, I will raise the score to a borderline accept.**

Thanks so much again for the time and effort in our work. May I know if our rebuttal addresses the concerns? If there are further concerns or questions, we are more than happy to address them. Thanks again for taking the time to review our work and provide insightful comments.

Thank you for your professional feedback. We believe our work does not involve any over-claiming. Firstly, "epipolar-free" and "geometry-free" are two distinct concepts. In our method, "epipolar-free" means that epipolar line sampling is not required, but it does not mean that we completely avoid using geometric information. In fact, our Iterative Cross-view Gaussians Alignment method relies on the warp transformation formula from the known geometry, as shown in Eq. (5) of the manuscript. In contrast, methods like SRT and GS-LRM are entirely geometry-free, meaning they do not use geometric information at all, making them true "geometry-free" methods. Our "epipolar-free" approach is more akin to a semi-geometry approach, so SRT and GS-LRM do not fall under the category of epipolar-free methods as proposed in this paper. We will further clarify the distinction between epipolar-free and geometry-free methods in the related works section of the revised version. Secondly, at this stage, unposed GNVS tasks and GNVS tasks with known poses are two different tasks. Here are three reasons to illustrate this: 1. Although pose-free GNVS methods can be applied to GNVS tasks with known GT poses, the current state-of-the-art methods, such as LEAP or PF-LRM, struggle to match the accuracy and efficiency of contemporaneous GNVS methods with known GT poses. These pose-free methods mainly compare themselves with known GT pose methods like pixelNeRF (CVPR 2021). 2. Pose-free GNVS methods are currently limited to simple object-level datasets (e.g., DTU or Omniobject3D). In contrast, our method not only conducts experiments at the complex scene level (RE10K) but also generalizes directly to object-level datasets (see Figure 1 and Table 1 in the rebuttal PDF). 3. The primary challenges of unposed GNVS methods and GNVS methods with known poses are different at this stage. Unposed GNVS methods need to address how to implicitly learn geometric relationships or recover camera poses without any known 3D priors. They often reduce task complexity through clever structured feature representations (e.g., Learned 3D Neural Volume in LEAP and Triplane in PF-LRM), but this comes at the cost of reduced model generalization. On the other hand, GNVS methods with known poses focus more on generalization ability and handling larger baselines. Our epipolar-free method is designed to enhance performance on larger baselines: eFreeSplat significantly outperforms SOTA GNVS methods in challenging areas such as regions with smaller viewpoint overlaps.