DarkSAM: Fooling Segment Anything Model to Segment Nothing

Summary

This paper introduces DarkSAM, a prompt-free universal attack framework against the Segment Anything Model (SAM) in a quasi-black-box setting. The framework consists of a semantic decoupling-based spatial attack and a texture distortion-based frequency attack. While SAM uses geometric prompt inputs to guide segmentation of critical objects within images, DarkSAM disrupts these processes by decoupling the object features of images in both spatial and frequency domains using a universal adversarial perturbation (UAP). In the spatial domain, it scrambles SAM’s decisions by destroying the features of the foreground and background of the image separately. In the frequency domain, it decomposes images into high-frequency components (HFC) and low-frequency components (LFC), increasing the dissimilarity in the HFC of adversarial and benign examples while maintaining consistency in their LFC. Experiments are conducted on four public segmentation datasets (ADE20K, MS-COCO, CITYSCAPES, and SA-1B), with 100 images used for UAP generation and 2,000 images for testing for each dataset. Victim models include the pre-trained SAM, HQ-SAM, and PerSAM with the ViT-B backbone.

Strengths

1. The paper is well-written and easy to follow. 2. The specific design of the shadow target strategy is tailored to SAM for prompt-based segmentation, which is unique compared to adversary attacks against traditional segmentation pipelines. 3. The attacking results are impressive. All three models show very low segmentation performance across multiple datasets.

Weaknesses

SAM is a milestone work, and a series of follow-up studies have been proposed recently. However, this paper does not provide an up-to-date review in Section 2.1 and lacks comparison in Section 4, weakening its significance. For example: - SAM-based adversary attack: [1] Practical Region-level Attack against Segment Anything Models, CVPR 2024. - Other SAM-based models: [2] From SAM to CAMs: Exploring Segment Anything Model for Weakly Supervised Semantic Segmentation, CVPR 2024. [3] RobustSAM: Segment Anything Robustly on Degraded Images, CVPR 2024. [4] Matching Anything by Segmenting Anything, CVPR 2024. [5] FocSAM: Delving Deeply into Focused Objects in Segmenting Anything, CVPR 2024. [6] ASAM: Boosting Segment Anything Model with Adversarial Tuning, CVPR 2024. [7] BA-SAM: Scalable Bias-Mode Attention Mask for Segment Anything Model, CVPR 2024. [8] Open-Vocabulary SAM: Segment and Recognize Twenty-thousand Classes Interactively, ECCV 2024. [9] CAT-SAM: Conditional Tuning Network for Few-Shot Adaptation of Segmentation Anything Model, ECCV 2024. [10] Semantic-SAM: Segment and Recognize Anything at Any Granularity, ECCV 2024.

Questions

1. How does DarkSAM perform with variants of the SAM models? 2. Please include a comparison with [1] "Practical Region-level Attack against Segment Anything Models" (CVPR 2024).

Rating

5

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

The paper presents a limitation that DarkSAM is tailored to SAM and cannot operate on traditional segmentation models.

Summary

This work investigates adversarial attacks against Segment Anything Models (SAMs) and presents DarkSAM, the first universal adversarial attack designed for these models. DarkSAM leverages a single perturbation to effectively undermine SAM’s object segmentation capabilities across a variety of images and prompts. The authors conduct a comprehensive evaluation of DarkSAM across four datasets and three SAM variants (SAM, HQ-SAM, and PerSAM), covering attack performance, transferability, comparative analysis, and ablation studies.

Strengths

1. The paper introduces a unique perspective on adversarial attacks for prompt-guided segmentation models, which is a relatively unexplored area in the literature. The proposed DarkSAM method is innovative in its approach to decoupling object features for attack optimization. 2. The research question is well-defined, and the authors thoroughly compare DarkSAM with a multitude of established baselines. 3. The paper presents both qualitative and quantitative results, effectively demonstrating the impact of DarkSAM. These results provide a thorough assessment of its performance across various conditions

Weaknesses

1. It is recommended that the authors further supplement the experimental section with relevant analyses, such as explaining why the spatial domain attack is more critical than the frequency domain attack within the proposed framework. 2. In this paper, the usage of mIoU and ASR appears to be analogous, with both metrics conveying the same information. Could the authors provide insight into the justification for employing both metrics concurrently? 3. In Figure 7, the visualization of segmentation masks is notably dark, impeding discernibility for the reader. The authors should consider increasing the brightness of these images or employing more vivid colors.

Questions

1. I agree with the authors' prompt-free approach to the attack on SAM. I am curious about the types of prompts that may be more advantageous in crafting effective UAPs during the attack generation process. Additionally, could the authors provide an explanation regarding the selection of prompts?

Rating

8

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

1. The paper relies on heuristic research and lacks a corresponding theoretical framework for analysis.

Summary

This paper introduces DarkSAM, a universal adversarial attack against the Segment Anything Model and its variants. DarkSAM aims to prevent these models from successfully segmenting objects within images. The experimental results demonstrate the effectiveness and transferability of the proposed method. I have read the response of the authors and comments of other reviewers, I decide to keep my weak accept score.

Strengths

1.This paper introduces a new universal adversarial attack framework for prompt-guided image segmentation models. 2.The combination of spatial and frequency domain attacks is a sophisticated approach that demonstrates a good understanding of SAM. 3.This paper is well-written. Following the introduction, I can easily understand the goal of this paper.

Weaknesses

1.The related work can be improved. This paper could benefit from an expanded discussion on adversarial attacks targeted at traditional segmentation models. 2.Lack of specific explanation. This method is novel and interesting but I’m curious about the reason why it works. What is the exact process for determining random prompts, and does this method ensure coverage of all potential attack targets? 3.The experimental results lack error bars. Repeating the experiments a few times and reporting the results with error bars would make the findings more convincing.

Questions

See Weaknesses

Rating

6

Confidence

5

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors adequately addressed the limitations and potential negative societal impact of their work.

Response from the reviewer

After carefully reviewing the comments from the other reviewers and the author's rebuttal, I find that all of my concerns have been adequately addressed. Therefore, I have decided to raise my score to 8.

Responses to Ethics Reviewer 5Dhu

Thank you for your valuable suggestions! In fact, we have already discussed the negative social impacts of adversarial attacks on SAM in the "Conclusions, Limitations, and Broader Impact" section of our manuscript (Lines 316-318). We further outline several representative scenarios: 1. **Medical Image Segmentation**: Attackers could craft adversarial examples to deceive SAM-based medical segmentation systems, potentially allowing patients to bypass disease screenings. 2. **Autonomous Driving**: Attackers might exploit SAM to create physical-world adversarial examples, thereby deceiving SAM-based autonomous driving systems and leading to severe traffic accidents. 3. **Surveillance Recognition**: Attackers could use SAM to generate adversarial examples that evade SAM-based surveillance systems for malicious purposes. We advocate for further research on defenses against adversarial attacks for SAM, such as adversarial fine-tuning of SAM using the adversarial examples generated by our proposed DarkSAM to enhance its robustness. We will include the above discussion and expand it to cover a broader impact in the revised version. We hope these responses adequately address your concerns, and we look forward to receiving further feedback.