Thanks for comments. Response to follow-up questions
**38HN.F1 - Questions: Why does the method perform worse than COMUS?**
As the reviewer correctly points out, our method underperforms COMUS in mIoU metrics on Pascal VOC 2012(43.3% versus 50.0%).
However, as also agreed by Reviewer 8Kba, we argue that, in comparison to our method, there are two additional components in COMUS possibly attributing to the performance differences and unfair comparisons. First, COMUS employs two pre-trained saliency detection architectures DeepUSPS (Nguyen et al., 2019, NeurIPS, https://arxiv.org/abs/1909.13055) and BasNet (Qin et al., 2019, CVPR, https://arxiv.org/abs/2101.04704) in addition to DINO. The saliency detection model requires an additional MSRA image dataset (Cheng et al., 2015, TPAMI, https://arxiv.org/abs/2101.04704) during pre-training. Thus, compared to our cyclic walks, COMUS indirectly relies on extra useful information. Second, COMUS uses Deeplab-v3 as its backbone for predicting semantic segmentation masks. For better segmentation performances, Deeplab-v3 extracts large-resolution feature maps from images and applies Atrous Spatial Pyramid Pooling for aggregating these feature maps over multiple scales. In contrast, our method extracts low-resolution feature maps with DINO and is not specifically designed for unsupervised semantic segmentation. Yet, it is remarkable that our method still achieves comparable performances as COMUS in unsupervised semantic segmentation tasks.
In addition to the comparable performance with COMUS in unsupervised semantic object segmentation, our method is capable of parsing the entire scene on the image into semantically meaningful regions, which COMUS fails to do. These include distinct backgrounds, such as sky, lake, trees, and grass. For example, in Fig 3(b) in the main body, our method successfully segments backgrounds, such as the lake, the trees, and the sky. However, COMUS fails to segment background elements. For example, in Column 4 of Fig 1 in the paper (https://arxiv.org/abs/2207.05027), COMUS only segments the foreground ships and fails to segment the lake and the sky. Segmenting both salient objects and other semantic regions is essential for many computer vision applications. This emphasizes the importance and unique advantages of our method.
In the final version, we will highlight the differences between COMUS and our method and discuss the advantages of our method over COMUS in Section 2.
**38HN.F2 - Questions:Lack of comparisions on without using pre-trained features.**
We thank the reviewer for the recommendation. In the final version, we will highlight that both DINOSAUR and our method use the pre-trained frozen DINO in Table 1.
Note that the results of DINOSAUR reported in our paper deviate slightly from the results reported in the original DINOSAUR paper (Seitzer et al., 2023, ICLR, https://arxiv.org/abs/2209.14860). This is because the code for DINOSAUR was not available before the deadline for Neurips submission. Hence, we strictly followed the model specifications in the original paper, re-implemented DINOSAUR on our own, and reported the results of our re-implemented version. These results include the performances of our re-implemented DINOSAUR on Birds, Dogs, Cars, and Flowers datasets, which were missing in the original DINOSAUR paper. We will release our re-implementation code of all the baselines upon publication.
As the reviewer pointed out, in the table below, we now copied the exact same results in the original DINOSAUR paper. For easy comparison, we also copied the results of our re-implemented DINOSAUR and our method from our paper. The ARI-FG metric performances in (mean+/- standard deviation) are reported.
From the results, we observed that our method outperformed the re-implemented DINOSAUR and performed competitively well as the original DINOSAUR in all the experiments. Note that in comparison to DINOSAUR, our method does not require decoders for reconstruction and hence, our method has fewer parameters, less GPU memory usage, converges faster during training, and achieves faster inference speed as indicated in Section 5.5.
In the final version, we will include this table, dedicate a short paragraph to highlight the differences between our method and DINOSAUR and discuss the advantages of our method over DINOSAUR.
| |Pascal VOC 2012|COCO 2017| MOVi-C|MOVi-E|
|:-------:|:----------------------:|:----------------:|:---------:|:---------:|
|Original DINOSAUR|24.6±0.2|40.5±0.0|67.2±0.3|64.7±0.7|
|Re-implemented DINOSAUR|27.5±0.2|36.2±0.7|64.0±0.5|62.4±0.7|
|Our method|29.6±0.8|39.7±0.8|67.6±0.3|64.7±0.7|