ProEdit: Simple Progression is All You Need for High-Quality 3D Scene Editing

Summary

This paper presents a task decomposition method that achieves more robust 3D scene editings. The main contribution is the concept of decomposition of the desired task (represented by a prompt) and the adaptive 3D Gaussian Splatting training process. The edited appearance and geometry using decomposed tasks are better than the results generated by previous methods.

Strengths

The strengths of this paper are: - The idea of decomposing the task is interesting and useful. - The results demonstrated in the paper is convincing to support the claim.

Weaknesses

The weaknesses of this paper are: - The simple linear decomposition scheme do not provide any semantic meaning, which is a bit hard to evaluate the result of each subtasks. - Moreover, there is no results (except fig. 1) of each subtask.

Questions

- It is unclear can the method do more object manipulation instead of just appearance editings. For example, it would be nice to show more examples for ScanNet++ scenes that involves moving objects or adding objects. Current results are mostly just change the appearance. - I think it is important to show and evaluate the results of each subtask, instead of just showing the final results in the main paper. (e.g., use a more complex task and test the alignment between the results of each subtask and the human task decomposition results). I understand this is still a limitation of simple linear interpolation scheme, but the results will be very meaningful and helpful imho for future research.

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

I already discussed the main limitation, i.e., the task decomposition in the previous section.

Summary

This paper presents ProgressEditor, which decomposes the 3D scene editing task into multiple subtasks and progressively modifies the scene which is represented by 3D Gaussians. The subtask decomposition is defined as the linear interpolation of the encoding of the editing prompt. Given the editing instruction, the proposed approach recursively searches for the proper subtask decomposition, so that the difficulty of each subtask is uniformly distributed. Then it progressively completes the subtasks with the proposed adaptive Gaussian creation strategy and finally obtains the edited scene.

Strengths

The strengths of this paper include: (1) It proposes a novel idea that decomposes the 3D editing task into multiple subtasks and progressively completes the editing. (2) It provides an adaptive 3DGS tailored to the progressive 3D editing framework, which is able to refine the 3DGS more efficiently. (3) The proposed approach generates high-quality editing results with clear texture and precise geometry.

Weaknesses

The weaknesses include: (1) The experimental evaluation is insufficient to validate the effectiveness of the proposed approach with the lack of quantitative assessment. (2) It lacks the ablation study on some technical designs described in the method section. Please see my questions below.

Questions

The key of the proposed approach is to decompose the 3D scene editing tasks to alleviate the multi-view inconsistency problem during the distillation process. However, what about the consistency of each subtask? The instruction encoding is not necessarily linear. So the inconsistency still exists in each subtask. Although each subtask should have less inconsistency compared to the original task, would the progressive editing (multi-stage editing) introduce additional burden to the process? Line 194-196 describe the subtask scheduling, where it adds additional subtasks r_0 and r_n. There should be an ablation study to validate the necessity of this setting. There should be more technical details about the Gaussian creation strategy described in lines 229-231. It is difficult to measure the advantage of the proposed method compared to the other alternatives. It seems the generated results of the proposed method preserve the feature of the person better in Fig.3. But it’s hard to say that it generates the results with more geometry editing (line 283). A quantitative evaluation should be reported to validate the effectiveness of the proposed method. In addition, a comparison of the time cost of each method should also be presented.

Rating

6

Confidence

4

Soundness

2

Presentation

3

Contribution

3

Limitations

This paper includes a discussion of the limitations of the proposed approach. It's better to present some failure cases for a better understanding.

Summary

This work focuses on instruction-based 3D scene editing. It proposes a progressive editing framework by decomposing the complex editing task into different subtasks based on the difficulties. In this way, it could ensure multi-view consistency in each easy subtask and finally obtain consistent editing for the whole task.

Strengths

1. The idea of decomposing a difficult task into several easy subtasks is interesting and makes sense. This can avoid inconsistent multiview edits based on a complex instruction. 2. The figure of pipeline clearly illustrates the methodology and motivation. Based on the visualization results, the proposed method demonstrates superior editing effects compared to the baseline methods.

Weaknesses

1. I have some doubts about the main technique of the article. This work defines sub-tasks of different difficulties by weighting instruction prompts and empty prompts with r. I am uncertain whether the editing difficulty is sufficiently sensitive to the weight r. The authors should provide 2D multiview editing results for different r to illustrate that as r increases, the inconsistency across multiple views also increases, indicating a rise in editing difficulty. 2. The illustrations in Fig.3 are not clear. I don't know which row/column corresponds to which method. Please clarify this in the response. 3. This paper does not provide a quantitative comparison with baselines. 4. Most of the results are based on human faces. The edits conducted on Scannet scenes are style transfer, which does not require high levels of consistency. It would be better to give more visualizations of outdoor scenes such as the 'bear' and 'garden', etc.

Questions

Please refer to the weakness. My major concern is the reasonability of defining the difficulties based on the weight r.

Rating

6

Confidence

5

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors have discussed the possible limitations and social impacts.