NeuRodin: A Two-stage Framework for High-Fidelity Neural Surface Reconstruction

Summary

This paper focuses on improving SDF-based volume rendering and proposes a new pipeline to address issues stemming from SDF-to-density conversion and geometric regularization. First, it changes a global scale parameter to local adaptive value, allowing more flexible density values to be converted from SDF. Second, the method proposes a novel loss function to align the maximum probability distance in volume rendering with the zero-level set of the SDF representation. Third, the paper claims that SDF regularization may be too strong to allow flexible topological changes and thus proposes a two-stage training process. The coarse stage operates similarly to a density field without strong constraints, while the refinement stage then encourages enhanced smoothness.

Strengths

1. The proposed improvements are well-motivated and technically sound in general. 2. The paper is well-written and easy to follow overall. It's great to have Figure 3 for illustration. 3. The paper appropriately mentions related work that shares a similar idea and the problems of prior solutions. 4. The authors compare the proposed method on two datasets and demonstrate its superiority over several baseline methods. They also provide qualitative examples for the ablation study.

Weaknesses

1. It would be more convincing to include quantitative results and more qualitative examples for verifying the effectiveness of the proposed components in the ablation study. Currently, the ablation study is only conducted on a single example scene qualitatively. 2. The idea of using an adaptive scale is not quite new, as it is also seen in previous works [1]. It would be better to discuss the differences with existing works sharing a similar idea. Also, it would be better to provide more motivating examples or analysis on why an adaptive scale is important, such as when to use a large scale and when to use a small scale. 3. For the two-stage training, the motivation is to make the coarse stage more like a density field without strong constraints. The paper claims that "eliminating or downweighting any geometric constraints often results in an unnatural zero-level set." However, this point is not verified in experiments. It is also strongly recommended to add an ablation variant that does not have the eikonal loss (using estimated gradient) in the coarse stage. 4. It is common to use numerical gradients in calculating the eikonal loss. The benefit of sampling a step size should be analyzed and compared more thoroughly in the ablation studies. 5. While the variance of stochastic gradients is understandable, it is unclear why this ensures stability for large features and flexibility for complex details. More explanation and analysis would be helpful. 6. The proposed pipeline is built upon many techniques from TUVR. It would be better to directly compare with TUVR in the experiments. 7. It would be beneficial to show results on the commonly used DTU dataset as well. 8. The paper needs more careful proofreading and polishing: (a) Line 25: "fails to intricate geometric details" (b) Line 28: "is produced by is by" (c) Equation (11), the symbol 'n' is not explained. (d) Equation (8), the symbol 'd' is not explained. [1] Wang, Zian, Tianchang Shen, Merlin Nimier-David, Nicholas Sharp, Jun Gao, Alexander Keller, Sanja Fidler, Thomas Müller, and Zan Gojcic. "Adaptive shells for efficient neural radiance field rendering." arXiv preprint arXiv:2311.10091 (2023).

Questions

1. In Equation (8), why does it only penalize a single point instead of a region or multiple points (both SDF > 0 and SDF < 0)? 2. Line 273: What does "prior knowledge in terms of SDF" refer to? 3. Why would optimizing the color conditioned on the normal restrict topological change?

Rating

5

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

Yes, it's mentioned in the supplementary material.

Summary

The paper improved NeRF based surface reconstruction from two aspects: adaptive scale $s$ and a bias correction loss to reduce the bias, which encourages the SDF becomes negative after the maximum at $t^*$.

Strengths

The position $\mathbf{r}(t)$ based scale $s$ increases the degree of freedom, which has the potential to improve the accuracy. The bias correction loss encourages the SDF to be negative after the maximum weight point.

Weaknesses

The bias correction looks like partially. It doesn't punish negative SDF before $t^*$, so it is not completed. The experiments is not comprehensive. TUVR is not compared, which also aims to reduce the bias.

Questions

While training $\mathcal{L}_{bias}$, how to evaluate $t^*$? Perhaps, it is an iterative procedure to obtain the maximum value, but can you embed the iterations into training? How to avoid the maximum $t^*$ behind the zero iso-surface? Why only use F-score for validation? What about other scores, such as Chamfer distance. NeRF based training is slow, but the training times are not reported in the paper. Does it become slower due to the bias correction?

Rating

5

Confidence

5

Soundness

3

Presentation

3

Contribution

3

Limitations

In the limitation section, it needs to more details about reconstruction of thin structures, sharp features, etc.

Summary

This paper introduces an innovative two-stage framework, NeuRodin, for neural surface reconstruction that significantly improves upon previous SDF-based methods. Locally adaptive parameters for SDF-to-density conversion and a novel explicit bias correction are introduced to enhance the fine reconstruction of SDF surfaces.

Strengths

1. The paper thoroughly elaborates on the shortcomings of current SDF-based approaches and provides insightful and impactful plug-and-play solutions, such as SDF-to-density conversion and explicit bias correction. 2. The paper is clearly written with detailed explanations of the methodology, the challenges addressed, and the solutions proposed. 3. Strong experimental results demonstrate that the proposed framework can generate high-quality surface reconstruction results.

Weaknesses

1. The two-stage optimization process proposed in the paper seems somewhat cumbersome, and finding a balance between over-regularization and fine reconstruction is challenging. This detailed design somewhat weakens the depth of discussion on the essential issues of SDF reconstruction. Easily portable modification modules, such as explicit bias correction and SDF-to-density conversion, could have a more far-reaching impact. 2. The difference between the ideal situation discussed by the authors in Explicit Bias Correction and the SDF distribution in actual training seems to have a similar mechanism to the convergence difficulty of shape adjustment discussed in [1]. Is it possible to avoid the biased situation shown in Figure 3 by introducing depth or other supervision information? 3. In the experimental section, Table 2 does not explain the measurement indicator of the data. Combined with Appendix 5, it can be inferred that the indicator is the percentage F-Score. 4. There is a lack of comparison of image reconstruction results. Since color is used in the training process and the baseline method in the paper includes image reconstruction comparisons, why does this paper not provide qualitative or quantitative results of image reconstruction? [1] Yang, Huizong, et al. "Stabilizing the Optimization of Neural Signed Distance Functions and Finer Shape Representation." *Advances in Neural Information Processing Systems* 37 (2023).

Questions

Refer to the weaknesses section.

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors discuss the limitations of the proposed model.

Summary

This article introduces NeuRodin, a Signed Distance Function (SDF)-to-density-based neural surface reconstruction method. The author summarizes the two main factors, SDF-to-density representation and geometric regularization, which cause low-quality performance in SDF-based methods and improve them in the pipeline. They argue that the widely used global scale parameter may cause the identical density in the same SDF level set. The paper attempts to solve this problem by using a non-linear mapping to obtain this scale parameter and adding a bias of density. For the geometric regularization, they mentioned some reasons such as the Eikonal loss and smoothness constraints that caused oversmoothness. Stochastic-step numerical gradient estimation and two-step training strategy are employed for over-regularization issue. The results overperform some baselines and ablation studies show the effectiveness of the modules.

Strengths

The mentioned factors theoretically may cause the low-quality performance. The pipeline is clear, and theories are reasonable. The strategies and improvements enhance the performance of the model. The shown depth maps examples also improved with the proposed strategies. This could be useful for other depth estimation tasks.

Weaknesses

The authors only provide F-score for the evaluation metrics except for some single scenes in appendix. It will be good to provide more such as PSNR, SSIM, etc. There are some minor mistakes in Table 5. The underline format in caption is not consistent with the italics format in the table. Is it possible to compute the proposed method under gird resolution of 2^22 just as nerualangelo did? This could be a fairer comparison in table 1. It will be good to discuss the runtime compared with other baselines.

Questions

Is it possible to compute the proposed method under gird resolution of 2^22 just as nerualangelo did? This could be a fairer comparison in table 1. It will be good to discuss the runtime compared with other baselines.

Rating

6

Confidence

2

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors only provide F-score for the evaluation metrics except for some single scenes in appendix. It will be good to provide more such as PSNR, SSIM, etc.

Please let us know if your concerns have been addressed

Dear Reviewer o1gj, Thank you again for your review. We hope that our rebuttal could address your questions and concerns. As the discussion phase is nearing its end, we would be grateful to hear your feedback and wondered if you might still have any concerns we could address. Thank you for your time.

Please let us know if your concerns have been addressed

Dear Reviewer EWQD, Thank you again for your review. We hope that our rebuttal could address your questions and concerns. As the discussion phase is nearing its end, we would be grateful to hear your feedback and wondered if you might still have any concerns we could address. Thank you for your time.