Flatten Anything: Unsupervised Neural Surface Parameterization

Summary

The authors introduce the Flatten Anything Model (FAM), an unsupervised tool for UV mapping 3D geometries. FAM consists of four main components: 1. **Deform-Net**: Adjusts 2D points in an input grid. 2. **Warp-Net**: Converts these 2D points into 3D space. 3. **Cut-Net**: Predicts a seam along which the 3D mesh is cut for unwrapping. 4. **Unwrap-Net**: Projects the cut 3D points back into 2D space. Since the unsupervised model uses two-cycle consistency losses to ensure the mappings from 2D to 3D and back remain consistent. During inference, the UV mapping process utilizes the Cut-Net and Unwrap-Net modules in sequence. Unlike similar approaches, FAM generates a single, easily usable unwrapped patch and can also work with unstructured point clouds. It outperforms selected baselines on 3D mesh tasks.

Strengths

- The paper is well-organized and the narrative is clear, with each component having a clear purpose. The authors' efforts to replicate the standard procedures for creating a UV map are commendable. - The paper positions itself effectively within the current body of research. Initially, I noticed an absence of comparison with cutting SOTA methods such as Nuvo (or the widely-used xatlas). Nonetheless, the paper justifies this by clearly explaining why a comparison is not feasible. - Regarding performance, FAM outshines SLIM and matches FBCP-PC, which is designed specifically for point clouds, unlike FAM. - Dividing the pipeline into separate, understandable components simplifies the process of tuning the model and integrating it with different methodologies.

Weaknesses

- One can argue with the statement "... global parametrization, a more valuable yet much harder problem setting". Such a parametrization serves a different purpose than real-world applications. For instance, implementing this parametrization on a car or any technical CAD object is impractical, whereas an atlas of patches is more applicable. - Apart from that observation, it's challenging to identify any major flaws in this work. The resolution and clarity of some figures could be enhanced. For instance, the teaser dedicates excessive space to phrases like "complex topology". Instead, enlarging the independent figures would be beneficial. Moreover, Tables 1-3 should clarify whether the metrics are intended to be minimized or maximized. It is important to note that this comment does not affect my overall evaluation.

Questions

I reiterate the suggestions mentioned in the weakness for self-consistency: > Apart from that observation, it's challenging to identify any major flaws in this work. The resolution and clarity of some figures could be enhanced. For instance, the teaser dedicates excessive space to phrases like "complex topology". Instead, enlarging the independent figures would be beneficial. Moreover, Tables 1-3 should clarify whether the metrics are intended to be minimized or maximized. It is important to note that this comment does not affect my overall evaluation. Discussing the potential applications of this method, particularly for certain types of objects or its applicability to meshes derived from NeRFs [1] or NeuS [2], which are inherently noisy, would be highly beneficial. [1] Mildenhall B, Srinivasan PP, Tancik M, Barron JT, Ramamoorthi R, Ng R. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM. 2021 Dec 17;65(1):99-106. [2] Wang P, Liu L, Liu Y, Theobalt C, Komura T, Wang W. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. arXiv preprint arXiv:2106.10689. 2021 Jun 20.

Rating

8

Confidence

4

Soundness

4

Presentation

3

Contribution

4

Limitations

The authors have not addressed the limitations of their method. It would be beneficial to include a separate paragraph, possibly in the supplementary materials, detailing the types of objects to which FAM is applicable, including its suitability for simpler CAD objects.

Summary

This paper proposes an unsupervised neural surface parameterization method, named FAM, which maps 3D surface points to adaptively deformed UV coordinates in the 2D parameter domain. Inspired by the actual physical procedures, the neural architecture includes several sub-networks for surface cutting, UV deforming, unwrapping, and wrapping, respectively. These sub-networks are assembled into a bi-directional cycle mapping framework, with carefully designed loss functions as constraints. The proposed FAM is the first method to achieve global free-boundary parameterization.

Strengths

1. The proposed FAM is the first method to achieve global free-boundary parameterization. Compared to existing methods, FAM can 1) directly deal with global parameterization, without needing manual effort for surface cutting; 2) map 3D points to an adaptively deformed 2D domain with free boundary, thus reducing distortions. 2. The design of several sub-networks and objective functions make sense to me. 3. The extracted cutting seams of the surface look reasonable.

Weaknesses

1. Missing analysis about the Cut-Net. From my point of view, Cut-Net is a very important module as it enables the joint learning of surface cutting without any supervision or manual effort. However, this module is not analyzed in ablation studies. Specifically, I am curious about whether this Cut-Net can be removed by directly mapping 3D surface points to 2D UV coordinates with the Unwrap-Net. If not, why can cutting seams be learned without any constraints on the intermediate P_cut? 2. In experiments on surfaces with disk-topologies or open boundaries, only one baseline SLIM is evaluated. I wonder why the authors do not evaluate the more recent DiffSR and Nuvo, since these local parameterization methods should also be applicable to such surfaces. 3. In experiments on surfaces with more complex topologies, only one baseline FBCP-PC is evaluated. The authors should also try to include other recent methods, even though they need manual surface cutting. Despite the unfair comparison, we can still see the gap and potential of the proposed automatic surface cutting. Besides, the authors can brute-forcely apply local parameterization methods without providing manual surface cutting if possible, therefore further demonstrate the advantages of the proposed automatic surface cutting.

Questions

Please refer to “Weaknesses” part for suggestions.

Rating

5

Confidence

4

Soundness

3

Presentation

2

Contribution

3

Limitations

The authors have clearly demonstrated the limitations of this work.

Summary

This paper proposes a novel neural-network based optimization framework for obtaining surface parameterization of arbitrary 3D meshes. Comparing to traditional methods like SLIM, this work can work for possibly low quality meshes of arbitrary topology. The core of the proposed pipeline are 4 simple MLPs: Given a point cloud possibly sampled from a 3D mesh, (1) the Deform-Net first deforms a square 2D uv grid to an irregular 2D shape with free boundaries to facilitate lowering the parameterization distortion. Then (2) the Wrap-Net lifts the deformed 2D grid to 3D to match the shape of the input point cloud. (3) The Cut-Net then tries to deform the lifted points to cut the shape to a disk topology. After this (4) the Unwrap-Net further flattens the points in the 2D uv plane. To better train the model, the paper also proposes an interesting cycle-consistency loss that aligns the 2D->3D->2D process and 3D->2D->3D process.

Strengths

- **Simple method to solve a challenging problem**. Traditional algorithms usually involve designing sophisticated energy functions where this method is straight forward and effective. It seems the method is also easiler to implement using python deep learning libraries than the traditional methods that typically requires C++ programming. - **Good results and support arbitrary topology**. The proposed method can be applied to arbitrary shapes and achieve similar or better results than traditional methods like SLIM. It also works for shapes with very high genus (i.e. the 'complex topology' example in Fig. 1).

Weaknesses

- **Advantages over multi-chart methods are unclear**. Although this work targets at a global parameterization, its advantage over multi-chart based local parameterization is not thoroughly discussed. This is especially important to convince people that this work is indeed more useful than works liek NUVO in some scenarios. To me, the NUVO method already does a pretty good job at unsupervised learning for surface parameterization using neural networks. Although there is only a third-party implementation available (https://github.com/ruiqixu37/Nuvo), I believe some qualitative analysis is needed to highlight the pros/cons of the two different paradigms. - **Missing discussion on the very related work of 'OptCuts', which is also a baseline of Nuvo.** - **It's unclear how the Cut-net is effective.** The purpose of cut-net is to get the seam, but the network design is just a simple MLP to deform the lifted 3D points. Also, there are no losses functions specifically designed for the cut-net. It is unclear how the network will do a reasonable cut to the surface. So I doubt that it is actually a neccessary component. Maybe the Unwrap-net along is good enough.

Questions

What is the actual advantage of this work over multi-chart based approach like Nuvo? What would be the case that the proposed method fails but the traditional methods like SLIM work? For example, the Fig. 10 of SLIM shows an example of Tutte’s embedding as a stress test. It works very well. Does the proposed method also work for this case? If not, why? Why Cut-net is a neccessary component?

Rating

6

Confidence

4

Soundness

3

Presentation

2

Contribution

3

Limitations

The authors discussed the limitation in tthe conclusion session.

RE: Rebuttal

I thank the authors for their response. Application of the suggestions discussed in all the reactions will improve the readability of the approach. The paper may have a high impact on the field upon acceptance.

Thank you

Thanks for the detailed response. My questions are largely resolved. Looking forward to the code release. I'm excited to try it out on my 3D models and see how it works.