Weakly-Supervised Concealed Object Segmentation with SAM-based Pseudo Labeling and Multi-scale Feature Grouping
Weakly-Supervised Concealed Object Segmentation (WSCOS) aims to segment objects well blended with surrounding environments using sparsely-annotated data for model training. It remains a challenging task since (1) it is hard to distinguish concealed objects from the background due to the intrinsic similarity and (2) the sparsely-annotated training data only provide weak supervision for model learning. In this paper, we propose a new WSCOS method to address these two challenges. To tackle the intrinsic similarity challenge, we design a multi-scale feature grouping module that first groups features at different granularities and then aggregates these grouping results. By grouping similar features together, it encourages segmentation coherence, helping obtain complete segmentation results for both single and multiple-object images. For the weak supervision challenge, we utilize the recently-proposed vision foundation model, Segment Anything Model (SAM), and use the provided sparse annotations as prompts to generate segmentation masks, which are used to train the model. To alleviate the impact of low-quality segmentation masks, we further propose a series of strategies, including multi-augmentation result ensemble, entropy-based pixel-level weighting, and entropy-based image-level selection. These strategies help provide more reliable supervision to train the segmentation model. We verify the effectiveness of our method on various WSCOS tasks, and experiments demonstrate that our method achieves state-of-the-art performance on these tasks.
Paper
Similar papers
Peer review
Summary
This paper proposes to tackle Concealed Object Segmentation tasks by generating high-quality pseudo segmentation masks using state-of-the-art vision foundation models--SAM. SAM takes prompts as provided sparse annotations (e.g. points or scribbles) to produce figure-ground pixel labels, which are used to train the segmenter. Yet, the task remains challenging as SAM is not robust to out-of-distribution images where objects visually blended with the surrounding environment. This paper proposes to infer and select high-quality pseudo labels using 1) multi-augmentation result fusion and 2) entropy-based label filtering. Additionally, this paper proposes a multi-scale feature grouping machinery to enhance the performance of the segmenter. The proposed method is demonstrated on multiple benchmark datasets and achieves good results.
Strengths
1. The motivation is elaborated well. It's easy to understand what the tasks and the challenges are. 2. The performance is good among most of the benchmark datasets.
Weaknesses
1. Although the motivation is illustrated clearly, this paper does not provide good review of the literature, given the fact that the vision community has made great success for weakly-supervised segmentation tasks. For example, I list some reference by searching through google scholar. * Weakly-supervised semantic segmentation: - Token contrast for weakly-supervised semantic segmentation. Ru et al. CVPR 2023 - Ts-cam: Token semantic coupled attention map for weakly supervised object localization. Gao et al. ICCV 2021. - Multi-class token transformer for weakly supervised semantic segmentation. Xu et al. CVPR 2022. - Learning affinity from attention: End-to-end weakly-supervised semantic segmentation with transformers. Ru et al. CVPR 2022. - Universal Weakly Supervised Segmentation by Pixel-to-Segment Contrastive Learning. ICLR 2021. - ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation. Lin et al. CVPR 2016. * Weakly-supervised instance segmentation - Weakly supervised instance segmentation using class peak response. Zhou et al. CVPR 2018. - Simple does it: Weakly supervised instance and semantic segmentation. Khoreva et al. CVPR 2017. - Boxinst: High-performance instance segmentation with box annotations. Tian et al. CVPR 2021. 2. Generally speaking, the proposed machineries--multi-augmentation fusion and entropy-based filtering, are not novel. These techniques are commonly used in weakly-supervised segmentation literature, e.g. AffinityNet[1]. The contribution of this paper is more like to employ SAM to tackle weakly-supervised concealed segmentation tasks. 3. I think the major contribution of this paper is not highlighted. If the authors believe multi-feature grouping module is the major contribution, then demonstrating on the concealed object segmentation is not enough. It would be essential to show its efficacy on other segmentation tasks. If the authors believe the employment of SAM is the major contribution, then I think the novelty of this paper is not impressive. The multi-augmentation fusion and entropy-based filtering tricks are well known in the literature. It would be nice to see what are the distinctive perspectives or challenges when deploying SAM on the tasks. Overall, the proposed method is reasonable but not novel enough. 4. The authors do not describe the limitations of the proposed method. [1]: Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation. Ahn et al. CVPR 2018.
Questions
1. How do you use scribbles as prompts to the SAM? It would also be nice to include a brief formulation about how SAM generate masks for self-completeness. 2. I think the formulation in Sec 3.2 might be wrong (as far as I understand). In my understanding, just like slot attention, the cluster prototypes $P$ aggregates pixel features with weighted-mean pixel-to-prototype attention. (See Eqn 1 and 2 in Slot Attention paper) * In Eqn 7, $A$ should be $A = \frac{1}{\sqrt{C}} K Q^\top$ instead * In Eqn 8, $D$ should be $D_{i, j} =\frac{\bar{A_{i, j}}}{\sum_l \bar{A_{l, j}}}$ * How to broadcast cluster prototype to 2D grid? * What is $S$ in Eqn 10? 3. I think it would be interesting to see what are learned in the cluster prototype at different granularity. It would be nice if we can see the visualization of feature similarity (or attention) among cluster prototype and grid features for $N_1$ and $N_2$.
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
2 fair
Contribution
2 fair
Limitations
The authors do not describe the limitations of the proposed method.
Summary
This paper tackles the problem of Weakly-Supervised Concealed Object Segmentation (WSCOS). The first strategy is using SAM, which is a vision foundation model for generic segmentation. To handle the miss-segmentation of SAM, the authors propose techniques using multi-augmentation and entropy-based scoring. In addition, to deal with the similarity challenge of FG and BG, Multi-scale Feature Grouping (MFG) is proposed. The overall experiments show that the proposed method achieves state-of-the-art performance.
Strengths
1. Using SAM is one of the promising directions in the field of segmentation. It is good to see a SAM-based weakly-supervised segmentation approach. 2. Entropy-based weighting and selection methods are technically sounding.
Weaknesses
1. It seems that the baseline of the proposed method is already quite higher than the existing methods. Could the authors provide the backbone architecture of the proposed method and existing ones, and match them for fair comparison? 2. SAM also can be used to refine the pseudo label. One of the most simple approaches is using the conventional WSCOS method and then refining the obtained pseudo label with SAM. Since the proposed method is integrated with SAM in a learning phase, it should achieve better performance than the refining strategy. Please provide the comparison in detail. 3. I want to check the performance of using SAM (PLW+ILS) alone, without MFG. I'm not sure but the gain of MFG can be overlapped with that of using SAM. 4. Inference using SAM is quite heavy in terms of both time and memory. This drawback is particularly pronounced when inferring images with multiple augmentations. This computational burden should be addressed in this paper.
Questions
Authors mention that directly using SAM sometimes miss-segment the confusing regions. Have the authors tried the segment-everything option of SAM? I'm not sure about it, but in such confusing cases, this option usually provides better results than using point (or scribble) prompts.
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
NA
Summary
This paper investigates Concealed Object Segmentation based on the recently proposed foundation model SAM, which provides dense annotation for learning. To alleviate the la-bel noise caused by imperfect segmentation results of SAM on concealed objects, they adopt pixel-level and image-level uncertainty-based strategies to weight predictions and select reasonable training samples. To tackle the intrinsic similarity problem in the task, they design a multi-scale grouping module to group features from different levels. Further, they conduct experiments on commonly used datasets, which demonstrates better performance.
Strengths
+ To alleviate the sparse-annotated challenge, this work utilizes the recently proposed foundation model SAM to augment the label, along with entropy-based acquisition strategies, to alleviate the impact of low-quality segmentation masks produced by SAM. + To tackle the inherent similarity problem in COD/COS, they introduce multi-scale features grouping technique to group features with different granularities. + To demonstrate the efficacy of the proposed method, they conduct experiments on several concealed datasets compared with recent methods and ablate their design components. + The proposed model achieves high performance from the experimental results.
Weaknesses
- ”Weakly-supervised” claim is not appropriate in this work. Since the segmentation mask generated by SAM is close to the full supervision setup but carries noise, instead of sparse as the weak-supervised task definition, such a claim here is inappropriate. - Lack of novelty. The proposed entropy-based acquisition strategies are classic methods used in active learning to sample informative samples and semi-supervised learning tasks to generate pseudo labels. Moreover, multi-augmentation fusion is also a popular technique in semi-supervised learning works. The aforementioned existing methods may refer to the following literature (but not limited to): a. Settles, Burr. ”Active learning literature survey.” (2009). b. Ren, Pengzhen, et al. ”A survey of deep active learning.” ACM computing surveys (CSUR) 54.9 (2021): 1-40. c. Iscen, Ahmet, et al. ”Label propagation for deep semi-supervised learning.” Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019. d. Berthelot, David, et al. ”Mixmatch: A holistic approach to semi-supervised learning.” Advances in neural information processing systems 32 (2019). - Literature review is not sufficient. There are more works related to the Concealed Object Detection/Segmentation task, which are not included and discussed the difference with the proposed method in the paper. - The mask generation method in Eq.5 is wrong. First, $\tilde{E_i}$ is an entropy mask. Its value ranges from [0, ln2] rather than [0,1]. In Eq.5, the author uses $(1-\tilde{E_i})$ to get the reverse mask is not the correct way. Second, the author regards $(1-\tilde{E_i})$ as a certainty for pixels. They should use $(1-\tilde{E_i})$ as weighting coefficients and multiply $(1-\tilde{E_i})$ to cross-entropy loss to calculate the weighted loss. However, the author directly multiplies certainty $(1-\tilde{E_i})$ to label $\tilde{M_i}$, which causes the pseudo mask to no longer be 0 or 1, but a decimal point. For example, on pixel in $\tilde{M_i}$ equals 1, but it will become 0.5 when $(1-\tilde{E_i})=0.5$. Using the label=0.5 to supervise the segmentation model does not make sense. Third, the author proposes the image selection value \hat{\alpha} to remove images with high uncertainty. However, they multiply the $\hat{\alpha}$ to the pseudo mask in Eq.5, which confuses the reviewer. Because this operation will generate an all-zero mask as the pseudo mask. Obviously, using the all-zero mask to supervise modeling training brings the wrong guidance. The authors need to repolish the theoretical part of the paper. - The experiment is not convincing. In line 241, the authors set two baselines SAM-S and SAM-P but only fine-tune the decoder part and use the automatic prompt generation strategy. This way can not show the generalization ability of SAM. The authors should adopt the SAM encoder as the backbone network, finetune the whole model and abandon the automatic prompt generation strategy. Only in this way can we understand the generalization of SAM to the COD task. Furthermore, by comparing the way of pseudo label generation and the way of encoder finetuning, the reviewer can determine whether we need better pseudo labels for COD or just a SAM backbone. - The writing is not clear enough, especially the use of symbols is confusing. For example, $\phi_p$ in the caption of Figure 3 is not displayed in the figure. "S" in line 204 is given without any explanation.
Questions
1. In the Supp., the authors describe that they adopt the encoder-decoder architecture utilized in a more recent work FEDER. But they do not compare the results with this model in their experiments. 2. The ablation in Table 4 does not show the superiority of their sophisticated design choices, where the margins among experiment results are narrow.
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
2 fair
Presentation
1 poor
Contribution
2 fair
Limitations
Please refer to the weakness part.
Dear Reviewer yhjR: Could you please check if authors' extensive rebuttal has addressed your concerns? The author-reviewer discussion period is closing soon. Thank you! AC
Summary
The submission presents a method for Weakly-Supervised Concealed Object Segmentation (WSCOS) to address the challenges of segmenting objects that are well blended with their surroundings using sparsely-annotated data. + The method introduces a multi-scale feature grouping module that groups similar features at different scales, promoting segmentation coherence and producing accurate results for single and multiple-object images. + The authors leverage the Segment Anything Model (SAM) and utilize sparse annotations as prompts to generate segmentation masks for training the model. + Additionally, the paper proposes strategies such as multi-augmentation result ensemble, entropy-based pixel-level weighting, and entropy-based image-level selection to mitigate the impact of low-quality segmentation masks and provide more reliable supervision during training. + Experimental results demonstrate the effectiveness of the proposed WSCOS method.
Strengths
This paper exploits the hot SAM for weakly-supervised COS, which is a good point. This paper is well-written and easy to read and understand. They verify the method on several interesting applications. Still, there have several concerns during my reviewing, please see the below sector.
Weaknesses
The performance improvements may come from the strong capability and prediction of SAM, but not from the proposed weakly-supervised strategy. The observed performance of SCOD+ does not significantly surpass that of SCOD when integrated with the proposed WS-SAM framework. Thus, the proposed modules do not seem to offer a promising and significant solution to this challenging task. The comparison might not be fair due to the introduction of an additional model (SAM) to generate weak labels. Could the authors elaborate on this, particularly in comparison to the other methods displayed in Table 1? Several of the latest and SOTA weakly-supervised semantic segmentation methods, such as [ref1] "Weakly Supervised Semantic Segmentation via Adversarial Learning of Classifier and Reconstructor", [ref2] "Out-of-Candidate Rectification for Weakly Supervised Semantic Segmentation", and [ref3] "Weakly Supervised Semantic Segmentation via Adversarial Learning of Classifier and Reconstructor" could be applied to this challenging task. Overall, this submission falls short of the standard for a top-tier conference due to its weak experimental validations. I am inclined to recommend borderline rejection in the first round.
Questions
See above concerns.
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Soundness
2 fair
Presentation
3 good
Contribution
2 fair
Limitations
No significant limitations.
Thanks for the rebuttal. I think the authors have done good job in the rebuttal. The feedback has addressed most of my concern, except Q2 and Q3. The author claims multi-augmentation fusion is novel from AffinityNet[52], yet, the statement is not true. In fact, multi-scale augmentation during inference is commonly used in segmentation tasks. For example, in [AffinityNet's public code](https://github.com/jiwoon-ahn/psa/blob/9493e59bef16687ecb4821387e38e6460857f508/infer_cls.py#L40-L45), they do aggregate CAM from different scales. SEAM[Ref1] imposes consistency regularizations among augmentations. Not to mention that, DeepLab[Ref2] uses multi-scale fusion to bootstrap testing performance (see Table 4). I'm not convinced that *multi-augmentation fusion* proposed in this paper is novel. On the other hand, demonstrating the efficacy of MFG only on the COS tasks seems not enough. In my understanding, I don't think COS is special from other image segmentation tasks. Instead, COS imagery are a special subset of common images. As a result, it would be a plus if the authors can demonstrate that MFG also works on more general imagery. [Ref1] Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation. Wang et al. CVPR 2020. [Ref2] DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. Chen et al.
Further response to Reviewer DmES (1/2).
**Q2: Novelties of multiple-augmentation fusion.** Our multi-augmentation fusion (MAF) strategy differs from AffinityNet [52] and DeepLab [53] in that we perform MAF in the **training phase**, while AffinityNet and DeepLab do that in the **test phase**. In addition, since our masks are generated from SAM when with scribbles as supervision, we propose to generate more masks for fusion by sampling different versions of points from scribbles. While SEAM [54] also employs augmentation in training, it, however, aims to enforce consistency between an image and its augmented views for the predictions. Instead, we employ multiple augmentations for result fusion. To further demonstrate the difference, we conducted an experiment replacing our multi-augmentation fusion with the augmentation technique utilized in SEAM [54]. We can see a noticeable performance drop after the replacement. This further evidences the advantages of our MAF strategy for the WSCOS task. |Scribble|$M\downarrow$|$F_\beta\uparrow$|$E_\phi\uparrow$|$S_\alpha\uparrow$| |-|-|-|-|-| |Ours+SEAM|0.044|0.683|0.843|0.766| |Ours+WS-SAM|0.038|0.719|0.878|0.803| We would like to stress that the differences between our method and the two methods, *i.e.*, AffinityNet [52] and DeepLab [53], are more than they look to be. Rather than directly borrowing and adapting techniques used by existing methods, the multiple-augmentation fusion strategy is particularly designed by considering the uniqueness of employing SAM to solve the WSCOS problem. We are rooted in the observations that upon feeding a sequence of images augmented from the same original source into SAM, this method yields multiple masks characterized by notable variations in shape. Notably, these divergent masks exhibit regions of overlap, as evidenced by Fig. 2 in the manuscript and Fig. 4 in the supplementary materials. Intriguingly, these overlapping areas consistently demonstrate reliable predictions by SAM, undeterred by image transformations, often aligning with accurately predicted foreground regions. Furthermore, these masks complement each other, such that some foreground regions missed by one mask can be found in other masks. This realization underscores the rationale behind our approach of fusing segmentation masks derived from diverse augmented images. Such fusion culminates in a consolidated mask that transcends the individual masks in terms of reliability due to its ensemble nature, encapsulating insights drawn from a spectrum of augmented images. In summation, our multi-augmentation fusion strategy is meaningful to the field of weakly-supervised COS. It effectively tackles the challenge of generating stable masks in concealed scenarios, setting a robust foundation to improve the segmentation performance. **Q3: Superiority of our MFG on general segmentation tasks** We prove that our MFG is a plug-and-play module in multiple segmentation tasks, including salient object detection (SOD), semantic segmentation, and concealed object segmentation (COS). In specific, we add our MFG to existing state-of-the-art segmenters in the corresponding tasks following the practice in our method and see whether or not the addition of MFG will result in performance improvement. Note that the new segmenters equipped with our MFG are still trained with their original training strategies without any modifications. - Salient object detection. We report the results in the *DUTS-test* dataset. In the weakly-supervised setting, MFG improves existing cutting-edge segmenters by **3.9\%** (TEL) and **4.9\%** (SCOD). For those segmenters trained with full supervision, MFG reinforces their performance by **3.2\%** (PGNet [55]) and **4.0\%** (MENet [56]). |Scribble in SOD|$M\downarrow$|$F_\beta\uparrow$|$E_\phi\uparrow$|$S_\alpha\uparrow$| |-|-|-|-|-| |TEL [34]|0.045|0.842|0.901|0.863| |TEL+MFG|0.041|0.869|0.920|0.874| |SCOD [29]|0.046|0.830|0.892|0.855| |SCOD+MFG|0.041|0.862|0.918|0.872| |Full in SOD|$M\downarrow$|$F_\beta\uparrow$|$E_\phi\uparrow$|$S_\alpha\uparrow$| |-|-|-|-|-| |PGNet [55]|0.027|0.903|0.922|0.911| |PGNet+MFG|0.025|0.923|0.941|0.920| |MENet [56]|0.028|0.895|0.937|0.905| |MENet+MFG|0.025|0.919|0.949|0.917| - Weakly-supervised semantic segmentation. Following AFA [57], we conduct experiments on *PASCAL VOC 2012* and *MS COCO 2014* datasets in the weakly-supervised semantic segmentation task to verify the superiority of our MFG, where the two datasets are abbreviated as *VOC* and *COCO* in the table. Under weak supervision, MFG boosts the performance of the original segmenters by **3.1\%** (AFA [57]), **1.4\%** (AEFT [58]), **1.7\%** (ToCo [59]), and **3.8\%** (OCR [60]), where OCR is deployed on AffinityNet [52]. |Weak in semantic (mIoU (\%))|*VOC\_val*|*VOC\_test*|*COCO\_val*| |-|-|-|-| |AFA [57]|66.0|66.3|38.9| |AFA+MFG|68.1|68.9|39.8| |AEFT [58]|70.9|71.7|44.8| |AEFT+MFG|72.0|72.8|45.3| |ToCo [59]|69.8|70.5|41.3| |ToCo+MFG|70.9|71.7|42.0| |OCR [60]|64.9|65.2|30.5| |OCR+MFG|67.2|68.0|31.6|
Further response to Reviewer DmES (2/2).
- Semi-supervised semantic segmentation. Following SemiCVT [61], the experiments on semi-supervised semantic segmentation are conducted on *VOC\_blender* and *Cityscapes* datasets with 1/16 labeled training data. As shown in the table, MFG generally improves the segmenters by **3.4\%** (GCT [62]), **3.7\%** (CPS [63]), **3.5\%** ($\text{U}^{\text{2}}\text{PL}$ [64]), and **2.1\%** (SemiCVT [61]). Those improvements are great in the field of semi-supervised semantic segmentation. |Semi in semantic (mIoU (\%))|*VOC\_blender*|*Cityscapes*| |-|-|-| |GCT [62]|70.9|66.8| |GCT+MFG|72.3|69.9| |CPS [63]|74.5|69.8| |CPS+MFG|76.2|73.3| |$\text{U}^{\text{2}}\text{PL}$ [64]|77.2|70.3| |$\text{U}^{\text{2}}\text{PL}$+MFG|78.9|73.8| |SemiCVT [61]|78.2|72.2| |SemiCVT+MFG|79.1|74.4| - Fully-supervised semantic segmentation. Following TSG [65], experiments are implemented on *Pascal Context* and *ADE20K* datasets. As reported in the table, MFG improves existing cutting-edge segmenters by **2.3\%** (MaskFormer [66]), **1.6\%** (SenFormer [67]), **2.6\%** (Mask2Former [68]), and **2.4\%** (TSG [65]). |Full in semantic (mIoU (\%))|*Pascal Context*|*ADE20K*| |-|-|-| |MaskFormer[66]|53.3|46.7| |MaskFormer+MFG|54.5|47.8| |SenFormer[67]|53.2|46.0| |SenFormer+MFG|54.1|46.7| |Mask2Former[68]|54.6|47.7| |Mask2Former+MFG|56.3|48.7| |TSG[65]|54.5|47.5| |TSG+MFG|55.8|48.6| - Concealed object segmentation. In **Table A3** in the attained PDF, MFG improves the performance of segmenters by **2.1\%** (TEL [34]) and **3.5\%** (SCOD [29]). We further evaluate the effect of our MFG in the COS task and report the results in the *COD10K* dataset of the COD task with full supervision. As shown in the table, segmenters equipped with MFG generally surpass their original versions by **3.2\%** (FGANet [15]) and **3.0\%** (FEDER [51]). Note that SegMaR (CVPR22) [17] overall surpasses LSR (CVPR21) [69] by **1.2\%**, the gains brought by our MFG are significant. |Full in COS|$M\downarrow$|$F_\beta\uparrow$|$E_\phi\uparrow$|$S_\alpha\uparrow$| |-|-|-|-|-| |FGANet[15]|0.032|0.708|0.894|0.801| |FGANet+MFG|0.030|0.725|0.917|0.813| |FEDER[51]|0.032|0.715|0.892|0.810| |FEDER+MFG|0.030|0.732|0.911|0.819| To sum up, the proposed MFG has exhibited remarkable efficacy across a spectrum of challenging tasks, encompassing semantic segmentation, SOD, and even highly intricate COS tasks. It is noteworthy that the versatility of our MFG extends to its compatibility with diverse supervision strategies, spanning weak, semi, and full supervision scenarios, thereby substantially elevating segmentation performance. This capacity is a direct outcome of MFG's intrinsic attributes, including the exploration of foreground-background region coherence and feature grouping across multiple hierarchical levels. This distinct feature grouping approach holds applicability to a wide array of image segmentation tasks. Notably, while the segmenters' feature extraction capacity might be comparatively modest in instances like weak or semi-supervised settings, MFG adeptly capitalizes on the extracted discriminative cues. It achieves this through a multi-scale grouping mechanism that effectively fuses unattended features with the extracted ones. Consequently, segmenters fortified with MFG are aptly equipped to mitigate the issue of incomplete segmentation by augmenting local correlations within individual objects. Concurrently, they facilitate the segmentation of multiple objects by actively seeking global coherence across multiple entities. This holistic approach culminates in the strengthening of segmentation performance in multifaceted scenarios. These assertions are notably evidenced in **Fig. A1**, which graphically substantiates the advantages conferred by our MFG in enhancing both local and global segmentation outcomes. We will add the related experiments to our revised version. [53] DeepLab, TPAMI, 2017. [54] SEAM, CVPR, 2020. [55] PGNet, CVPR, 2022. [56] MENet, CVPR, 2023. [57] AFA, CVPR, 2022. [58] AEFT, ECCV, 2022. [59] ToCo, CVPR, 2023. [60] OCR, CVPR, 2023. [61] SemiCVT, CVPR, 2023. [62] GCT, ECCV, 2020. [63] CPS, CVPR, 2021. [64] $\text{U}^{\text{2}}\text{PL}$, CVPR, 2022. [65] TSG, CVPR, 2023. [66] MaskFormer, NeurIPS, 2021. [67] SenFormer, BMVC, 2022. [68] Mask2Former, CVPR, 2022. [69] LSR, CVPR, 2021.
Thanks for the update. I think these comments address my concern in Q2 and Q3. The difference between AffinityNet, DeepLab, SEAM, and the proposed method is clearly illustrated in the response. I didn't notice that SAM is a mask proposal framework: given different augmentations, the proposals will be different. On the other hand, AffinityNet, DeepLab, and SEAM are image parsing framework. The segmentation do not change drastically w.r.t augmentations. It will be helpful if the authors can highlight the difference in the paper. Also, the efficacy of proposed MFG is clearly demonstrated in the comments, which not only works in WCOS tasks, but also other segmentations tasks. It is a general and effective modules. I increase my ratings to accept based on these strong results.
Thanks for recognizing the value of our work!
We thank the reviewer for recognizing the value of our work, *i.e.*, the proposed **WS-SAM framework** and **MFG module**, to the field of COS. We appreciate the suggestion and will further highlight the differences between our method and existing weakly-supervised approaches in the revised version.
After reviewing all the materials and discussions on this page, I believe the authors have made significant efforts to address my main concerns. I am now happy to update my rating to acceptance. Best of luck to the authors!
Thanks for recognizing the value of our work!
We extend our gratitude to the reviewer for acknowledging the significance of our contribution, namely the **WS-SAM framework** and the **MFG module**, within the domain of concealed object segmentation (COS). Your recognition is deeply appreciated and serves as validation of our efforts in advancing this critical area of research.
Thanks for the rebuttal. All of my concerns are appropriately addressed. Since the overall impact of this paper is not that huge, I'll remain my rating as borderline accept. But I still think this paper is good.
Thanks for recognizing the value of our work!
We express our sincere appreciation to the reviewer for acknowledging the substantive impact of our contribution, notably the **WS-SAM framework** and the **MFG module**, within the domain of COS and for maintaining a positive rating. WS-SAM offers the **first solution** to address the WSCOS task that is **orthogonal to existing methods** by leveraging the power of SAM. Moreover, our proposed WS-SAM framework and the plug-and-play MFG module can **significantly enhance the performance of existing state-of-the-art segmenters**. Notably, MFG holds applicability to a wide array of image segmentation tasks, including **concealed object segmentation**, **salient object detection**, and **semantic segmentation**. (See ''**Further response to Reviewer DmES**'' for more details.)
Thanks the author for their detailed rebuttal, particularly the experiments for the SAM model with full finetuning, which addresses my major concern. Based on these improvements, I am inclined to upgrade my rating to "borderline accept." The author's thorough response and the subsequent enhancements have strengthened the paper's quality and merit.
Thanks for recognizing the value of our work!
We wish to express our sincere appreciation to the reviewer for recognizing the substantial significance of our contribution, specifically the **WS-SAM framework** and the **MFG module**, within the realm of concealed object segmentation (COS). Your acknowledgment carries considerable weight and serves as a meaningful validation of our relentless efforts to advance this pivotal area of research.
Decision
Accept (poster)