Reconstruction of Manipulated Garment with Guided Deformation Prior

Summary

The paper aims to recover garments that are manipulated instead of worn. The method first generates the UV mappings from point clouds, followed by ISP to recover the complete mapping. A diffusion model is used to extract the deformation priors and guide the recovery from UV mappings to 3D mesh. Experiments show that the proposed method delivers lower reconstruction errors and outperforms the baselines.

Strengths

* This method is able to recover garments in a more general and complex poses. * The proposed model achieves robust performance as shown in the experiments.

Weaknesses

1. This method seems to be garment-specific design, or even topology-depend. For example, to recover the shirt and pants, one need to train different models to recover the garments, leading to limited generalisation abilities. 2. While the ground truth may include too much details, i.e. too many wrinkles, and even looks a bit noisy, the recovered garments are oversmooth. The proposed model fails to recover high frequency details of the garments.

Questions

1. Are the recovered garments meshes able to be used for further animations? 2. How to connect the edges in the recovered mesh from point clouds? Are the edges fixed or dynamically connected? Since the point clouds do not include any information about the connectivities, how to define the edges? 3. In the qualitative results, such as Figure 5, the recovered garments seem to be smoother than the ground truth. Is this because of the “auto smooth” option during rendering? Could you provide some visual results of the smoothed ground truth garments? 4. What is the averaged number of points for different garments? Is the model able to deal with large number of points? 5. While the proposed method is able to handle garments in more complex poses, is it possible to compare with other baselines using the garments worn by the human body, e.g. the quantitative and qualitative results on CLOTH3D dataset?

Rating

4

Confidence

4

Soundness

2

Presentation

2

Contribution

3

Limitations

Please refer to the weaknesses and questions.

Summary

The pape addresses the challenge of accurately reconstructing the 3D shape of garments that are manipulated rather than worn. The authors leverage the Implicit Sewing Patterns model and introduce a diffusion-based deformation prior to recover 3D garment shapes from incomplete 3D point clouds. The method maps these points to UV space, generates partial UV maps, and uses a reverse diffusion process to produce complete UV maps and 2D to 3D mappings. The approach demonstrates superior accuracy compared to previous methods, especially in handling large non-rigid deformations.

Strengths

The focus on reconstructing manipulated garments rather than worn ones addresses a significant gap in current research, as most existing methods assume garments are worn and thus have less complex deformations. Combining ISP with a diffusion-based deformation prior is a strong methodological contribution, enabling the modeling of complex deformations that were previously challenging to capture.

Weaknesses

The accuracy of the reconstruction heavily depends on the quality of the input point clouds. Incomplete or noisy point clouds might still pose a challenge.

Questions

Can this method handle noisy or highly sparse point clouds effectively, and have you tested its robustness in such scenarios?

Rating

7

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

While the method shows promise, its ability to generalize across a wide variety of garment types and materials without retraining is not fully explored.

Summary

This paper presents a method for reconstructing folded and crumpled garments from point cloud data. It uses the implicit sewing pattern (ISP) model to represent the 3D shape in 2D uv-maps. The proposed method converts a 3D point cloud to sparse uv-maps and corresponding masks for front and back side using an encoder structure followed by a MLP. The incomplete masks are filled and used to guide the completion of the uv-maps via a diffusion process. Finally, the deformed mesh can be recovered from the filled uv-map.

Strengths

This paper improves state-of-the-art reconstruction of point cloud data for folded garments in visual quality as well as 3D accuracy. Notably, this is done while no prior knowledge of the garment geometry is needed. The usage of a diffusion network to fill the sparse 2D data of the ISP model is a clever idea and matches the network characteristics very well.

Weaknesses

The comprehensibility of the paper could be improved by discussing the different parts of the pipeline in order and clearly pointing out the result of each stage and its purpose for the next stage. Some intermediate results for different scenes might be helpful to follow the pipeline.

Questions

How many points do the input point cloud contain? Did you test how many points are necessary and how accurate do they have to be to produce a high-quality reconstruction?

Rating

7

Confidence

3

Soundness

3

Presentation

2

Contribution

3

Limitations

The limitations are just mentioned very briefly. Some quantitative evaluation on the number of intersections in the reconstructed mesh or more animated reconstructions would show how large these limitations are. An analysis might even benefit the method as e.g. the number of intersections seems to be low based on the qualitative results.