NeuralGF: Unsupervised Point Normal Estimation by Learning Neural Gradient Function

Summary

The paper proposes a method to learn normals for point clouds using a neural representation. The idea is to perform a multi-step point moving and compute a series of losses to constrain the gradient of the field to describe shape locality with consistency. The paper proposes a large series of comparisons with the existing methods, showing promising results, although the training time is particularly slow. == POST-REBUTTAL == After the discussion, the authors addressed my concerns. From other reviews, I also see a general consensus for acceptance, and the general recommendations are about clarifying some minor aspects. The only negative reviewer seems unwilling to defend the rejection position and does not point to significant weaknesses. Hence, I decided to increase my score and lean toward acceptance. Authors already agreed to incorporate the suggested changes, which are indeed essential; especially, I would stress that the two figures that show partial sections of complete shapes should be clearly described so as to not be deceptive. I wish authors best of luck with their work.

Strengths

1) The paper obtained good reconstruction results; the shown shapes have varying numbers of different geometries, and the method seems to outperform the competitors. 2) While the method part is not straightforward to understand, the attached code and the implementation details should provide enough details for replicability.

Weaknesses

1) The paper does not convey a clear and precise message about its contribution. I am not an expert in this specific field, and I have quite a hard time understanding what makes this work different from previous ones. Accordingly to the conclusion, the main contribution seems in introducing a new loss/optimization schema. However, from the ablation study (which is also not easy to inspect, given the number of experiments and the lack of a best in bold), the "full" method is not always the clear winner, and it makes unclear the contribution of the losses. From my understanding, the main advancement is in providing multi-scale neighbour consistency, which helps orient all the normals in the same direction. I suggest clearly stating the main insights and adding more structure to 3.2 (e.g., with paragraph titles). 2) Some works seem missing: [A] proposes a smoothness regularization for implicit representations; [B] involves a differentiable Poisson Shape reconstruction. I think a proper discussion would be useful, and especially fostering this kind of discussion in the related works. For example, at the moment, the two paragraph end highlighting that the method is unsupervised, and achieves better performance than the previous works. But how is this obtained? What is the key aspect that enables such advancement and is not considered in the previous works (if any)? [A]: Implicit Geometric Regularization for Learning Shapes, Implicit Geometric Regularization for Learning Shapes, Gropp et al., ICML 2020 [B]: Shape As Points: A Differentiable Poisson Solver, Peng et al., NeurIPS 2021

Questions

A) From 3.2 I cannot completely understand how the method obtains the correct normal orientation between the two possible directions. Is it obtained thanks to the multi-scale neighbourhood size? B) From supplementary material, the training time seems dramatically slower than other methods. What is the main cause of that?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

Limitations are sufficiently discussed in the supplementary material

Summary

The paper introduces a method for estimating oriented normals from a given point cloud by utilizing a neural networkto model a sign distance field (SDF). The proposed approach involves training the SDF representation, which allows for easy querying of gradients at the positions of the point cloud. The method employs a set of loss functions that simulate an iterative process of moving query points to match target points based on the neural gradients. Experiments are conducted on unsupervised oriented normals estimation from input point clouds, which may contain noise, outliers, and density variations.

Strengths

- The authors clearly highlight the limitations of previous works, providing a good motivation for their proposed method. The promising results showcased in Figure 1 further support this motivation. - The introduction and related work sections are comprehensive, providing the necessary background for understanding the normal estimation task. This makes the paper self-contained. - The method is presented in a well-structured manner, starting with a high-level overview before delving into the details. This organization helps to keep the reader engaged and informed throughout the paper.

Weaknesses

- The mathematical definitions and notations in the method section need to be revisited. Some crucial definitions, such as $f_i$ in Equation 2, are mentioned without being properly defined, making Sections 3.1 and 3.2 harder to understand. Additionally, the notation $ \{ Q, G\} $ is confusing as the operation involving this set containing two sets of points is not clearly defined or highlighted. The notation $f_i^G$ also requires clarification. Furthermore, the unit gradient $\boldsymbol{n}$ mentioned in Line 164 lacks clear information about its 3D positions. - The authors refer to the post-processing step of computing the gradients of the learned SDF as "inference". However, "inference" typically refers to the process of applying a trained model on unseen inputs for generalization. This usage of terminology can be misleading and confusing. - Figure 2 does not fully capture the presented method as it introduces notations that are only defined in the text. It is unclear from the figure alone, even with the caption, what the input to the model is. - A comparison to other neural SDF methods that learn an SDF from an input point cloud, such as SAL (https://arxiv.org/abs/1911.10414) or IGR (https://arxiv.org/abs/2002.10099), is missing. These methods could be relevant for extracting normals easily, as in the suggested method. - The paper lacks a reference to SAP (https://pengsongyou.github.io/sap), which provides a scenario of reconstructing normals and a surface from an input point cloud. - It is recommended to include illustrations for the different losses mentioned, similar to the visualization in Figure 2 (i) at the bottom.

Questions

Please see concerns and questions in weaknesses. Few further questions: - Are the loss coefficients tuned for different noise levels, or are they fixed throughout the experiments? - Can the presented method also yield the complete negative solution, i.e., $-\boldsymbol{n}$, which is equivalent up to a global sign? Does this depend on the initialization or other factors?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

The authors discussed limitation, however the limitation discussion and figure is shown soley in the appendix.

Summary

In this work the authors present an unsupervised framework for predicting globally consistent, accurate normals given a point cloud as input. The crux of the method is to predict an implicit surface representation (a signed distance function to be exact) from the input point cloud by leveraging the fact that the gradient of the Neural SDF of surface points gives the surface normal at that point. The authors distinguish their method from existing similar approaches by overcoming drawbacks like lack of global consistency by designing a loss function that penalizes incorrect signed predictions and also considers multi-scale neighborhoods. Furthermore, they consider a multi-step iterative approach for refining their surface normal estimations. They show impressive results in terms of RMSE of unoriented/oriented normals across challenging datasets across different levels of sampling densities and noise.

Strengths

1. The proposed method achieves SoTA performance compared to other unsupervised methods on the PCPNet and FamousShape dataset. 2. In the presence of noisy input points, their method is quite competitive even against supervised baselines which is a big plus. 3. Accurately oriented point clouds are hugely sought-after in downstream applications like surface reconstruction and their results on single object surface reconstruction demonstrates the value of their method in this important application.

Weaknesses

1. Comparison with other implicit representations. At its core, the proposed method learns a Neural SDF for the shape represented by the input point cloud and the estimated normals are simply the gradients computed from this Neural SDF using automatic differentiation. As such I believe there should be more comparison drawn to similar methods like Neural-Pull [1], SIREN [2], SAL [3], and Neuralangelo [4]. In particular, [4] uses a finite-difference-based approach to estimate the surface normal during training and I would be curious to see if a similar technique can get similar results in lesser resources for normal estimation. 2. Hyperparameter tuning. Since the method is optimization based, one potential risk is mishandling the hyper-parameter tuning, which turns out to be very important. For example, how does the method choose the hyper-parameter for each shape? Does a different input point set require different hyper-parameters or one hyper-parameter can be used for a wide range of shapes? I think tuning hyper-parameter of a neural field fitting procedure can drastically change the performance, and arguably with proper hyper-parameter tuning, one can probably find a way to curate smooth solutions to different input instances. As a result, it's very essential for this kind of test-time optimization-based method to have a rigorous hyper-parameter tuning procedure reported to eliminate the risk of accidentally adding human judgment into producing the results. References: [1] Neural-Pull: Learning Signed Distance Functions from Point Clouds by Learning to Pull Space onto Surfaces. https://arxiv.org/abs/2011.13495 [2] SIREN. https://arxiv.org/abs/2006.09661 [3] SAL. https://arxiv.org/abs/1911.10414 [4] Neuralangelo: High-Fidelity Neural Surface Reconstruction. https://research.nvidia.com/labs/dir/neuralangelo/ [5] NKSR. https://research.nvidia.com/labs/toronto-ai/NKSR/

Questions

1. The density-based experiments are unclear to me. Could the authors please explain what the “gradient” and “stripe” settings exactly imply? The supplementary shows a few qualitative examples but can you explain how they were generated? It also seems that supervised methods perform comparably w.r.t. the proposed method. Could the authors give an intuition for why their method suffers from density variation but not on noisy point clouds (see Table 2 and Fig. 5, especially at low threshold angles)? Which hyperparameters can be tuned to address noise vs. density variation and what is the trade-off for the same? This would help with a better understanding of the authors’ contributions. 2. In the analysis of un-oriented point clouds, we can see that LRR is the second-best performing unsupervised normal estimation method. Yet, we do not see an application of LRR with techniques for introducing orientation like MST in the study on the estimation of oriented normals. Adding LRR + (some orientation method) type of methods to Table 2 and Fig. 5 would make the results a bit more stronger. Basically this can help address the question of why don't we just pick a SoTA unoriented unsupervised point cloud method and do some simple orientation method?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Limitation section in the supplementary. Potential other limitations can include shapes with open surface and shapes without volume.

Summary

This work proposes to learn neural gradient functions from point clouds to estimate oriented normal in an unsupervised manner. Specifically, this method introduces several loss functions to constrain query points to iteratively fit the underlying surface, which is defined by the sampled points. Meanwhile, the local gradients are incorporated into the surface approximation to measure the minimum signed deviation of queries, resulting in a consistent normal field associated with the surface. Lastly, some evaluations demonstrate the superior performance of the proposed method over existing approaches.

Strengths

1. This paper is well organized and nicely written. The presentation of motivation is clear and smooth. 2. The work is well motivated. The idea of learning neural gradients for normal estimation is inspiring. 3. Some visual and quantitative evaluation are promising.

Weaknesses

1. I am not sure why we need a distribution D. 2. I do not see any running efficiency statistics, which would be important for a fair assessment. 3. I think there is a missing comparison with [73]. Also, some challenging cases from [73] should be included.

Questions

1. I believe that the designed loss functions can facilitate query points to iteratively reach the moving targets and aggregate onto the approximated surface, thereby learning a global surface representation of the data. However, I am doubting that if the incorporate gradients can achieve a consistent normal field, especially handling some challenging cases. Is it possible to give more details? 2. Both point sets of Q and G are sampled from input point cloud, so in the test stage, how to compute the normal for all points?

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

4 excellent

Contribution

2 fair

Limitations

1. I do not see any failure cases in the paper. It is essential to show some failure cases for the reader to investigate the failure model to benefit future research.

Summary

The paper proposes a method that predicts oriented surface normals given a 3D point cloud. The method is based on SDF surface reconstruction. Given a potentially noisy point cloud, a SDF is fitted using an approach similar to Neural-Pull [45]. The surface normal at each point can then be computed based on the gradient of the SDF. Since a global SDF of the shape is recovered, the orientation of the predicted surface normal is also globally consistent. Additionally, the paper proposed various improvements on top of Neural-Pull to improve normal prediction quality: a multi-step movement strategy during optimization, and a multi-scale neighborhood size strategy.

Strengths

* The proposed method is unsupervised -- It is optimized on a per-shape basis and unlike [23,8,39,38], it does not need to be trained on a large collection of shapes, which also avoids any training-evaluation domain gap. * Unlike local fitting methods [8,82,37,15], the proposed method produces globally consistent surface orientation thanks to the use of a global SDF. * The proposed method achieved state-of-the-art performance on both oriented and unoriented surface normal estimation. It worked well especially for noisy point clouds. * The paper has included very comprehensive ablations to show the effects of various design decisions as well as the new components such as multi-step supervision and multi-scale neighbor selection.

Weaknesses

* The proposed method resembles many existing works on point cloud 3D reconstruction, such as SAL [6], Neural-Pull [45], Shape As Points (Peng et al.), NDF (Chibane et al.). In fact, recovering the normal can be considered as the side effect of surface reconstruction -- once the oriented surface is obtained, the oriented normal can be obtained naturally. As there is no comparison on surface normal quality with such methods in the paper, it is not clear if the proposed method has any significant benefit over these methods from surface reconstruction community. * Compared to feed-forward methods such as PCPNet [23], the proposed method requires per-shape optimization, which can be computationally expensive. * The paper will be easier to follow if it can devote some paragraphs to the connections between state-of-the-art surface reconstruction and normal estimation literature.

Questions

* How does the proposed method compare with previous works in terms of speed? * It is possible to directly repurpose surface reconstruction methods for normal estimation? How would they perform?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The limitation and societal impact of the paper is adequately addressed in the supplemental material.

More questions on hyper-parameter tuning

It’s still not very clear to me how a hyper-parameter is chosen for each dataset. Which metric do you use to tune the hyper-parameter? How do you select hold-out set?

Keep my rating

I would like to thank the authors for the response. The rebuttal has solved all my concerns, especially on its advantages over directly using surface reconstruction methods to recover surface normal -- it seems that the proposed method performed significantly better than generic surface reconstruction methods. I would like to retain my rating of weak accept. What prevents me from giving higher rating is mostly due to the exposition. As the proposed method is based on surface reconstruction methods, it will be helpful to compare and contrast the two, instead of trying to describe the proposed method as something new.

Post-rebuttal

I thank the authors for their answers to my concerns. I understand that the normal orientation can be recovered using the SDF, which separates the inside from the outside. However, in some of the shown examples, the concept of inside/outside is not well defined (e.g., Figure 3, Figure 7). How are these cases solved? Is the output sign ignored, and the normals are considered unoriented (and visualization is just illustrative)? At the moment, I do not have further questions, and I am considering increasing my score. I am looking forward to reading other reviewers' opinions.

post rebuttal

Thank you for addressing my main concerns and issues. I encourage the authors to add the mentioned clarifications regarding the notations and improve Figure 2 and the loss illustrations. The additional results presented in the rebuttal are compelling. They should be incorporated in the final submission, especially the comparison to other neural implicit representation methods that struggle to reconstruct from noisy point clouds. As most reviewers mentioned this requirement, it is clear that the paper would benefit from such comparison and discussion in the main paper. Based on these and the other reviews, I raised my rating to week accept. I believe the paper has a solid contribution, but it requires the mentioned amendments in order to be a valid submission.

Post Rebuttal

Dear Reviewer vh4j, We have provided a comparison with GCNO [73], used the challenging case of GCNO, and further clarified how incorporating gradients achieves consistent normals. In light of this, we would like to know whether you believe we have addressed your concerns, and if so we hope that you would be willing to increase your score. Thank you for your time, The Authors