Reply to Reviewer Gzhh's feedback at General Response
We sincerely appreciate reviewer *Gzhh* for checking our rebuttal. We address this remaining concern as below.
***
1. **Different training datasets used for quantitative evaluation.**
- Our DreamScene2D is designed to generate high-quality 3D scenes that match the input layout. To achieve this, we require a dataset with **high-quality 3D bounding box annotations** with accurate category labels for training.
- We therefore chose to train our DreamScene2D using the Hypersim [45] dataset. This choice of the training dataset is also *consistent* with ControlRoom3D [48].
- Other baseline methods use ScanNet as their training dataset. However, the bounding box annotations in ScanNet are of low quality (noisy and inaccurate), and cannot satisfy our requirements.
- While we understand the reviewer’s concern that using different training datasets might influence specific quantitative metrics, re-training the baseline models using the Hypersim dataset we employ is challenging and time-consuming. Additionally, we think that dataset selection is a critical design choice in developing these methods and should be considered during evaluation. For example, as mentioned above, we chose Hypersim for DreamScene2D specifically because of its high-quality 3D bounding box annotations. Therefore, using the exact same training data for comparison in our context may not be practical or necessary.
- In fact, comparing methods trained on different datasets is a common practice in generative model research. For example, ControlRoom3D [48] also compares with methods (MVDiffusion and Text2Room) trained on different datasets. Similarly, the well-known 2D diffusion model Instruct-Pix2Pix is compared with T2L and SDEdit, which are trained on other datasets.
- Importantly, our quantitative experiments focus on the CS, IS, and user study metrics, which evaluate the final generation results using either pre-trained models (CLIP and Inception) or participants’ judgments. These metrics do *not* require ground-truth scenes from the dataset and, therefore, are *agnostic* to the dataset.
- Therefore, even when using a different dataset for training, our quantitative comparison remains fair and reasonable with the current metrics and adheres to established practices.
2. **Not using FID metrics.**
- First, both our method and [48] focus on the overall quality of generated scenes and layout-following capability, unlike MVDiffusion which focuses on the quality of the diffusion image generation. Therefore, our method and [48] choose to report current metrics rather than FID.
- On the other hand, the value of the FID metric is dependent on the dataset. Since we use different datasets for diffusion finetuning, applying FID would result in unfair and inaccurate comparison.
***
If the reviewer has any follow-up questions, we are happy to discuss them.