Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Models

Summary

This paper proposes a (training-only) data poisoning attack against vision language models (VLM). The proposed method works similarly to a clean-label targeted data poisoning attack for image classification tasks, so it is designed to be stealthy in the generated poisoned image-text pairs. In addition to the traditional setting of label attack, the paper proposes a new threat model of “Persuasion Attack”, targeting a specific concept (e.g., junk food vs. healthy food) instead of a specific text phrase. Experiments show the proposed poisoning attack is highly effective with small poisoning rates.

Strengths

1. The proposed “Persuasion Attack” threat model is interesting, which is new. 2. The paper is well-written with a nice structure for readers to follow. 3. The proposed attack strategy is well-explained. 4. The experiments are very comprehensive and demonstrate the effectiveness of the proposed poisoning scheme against VLMs.

Weaknesses

1. It is unclear whether the attack is generalizable under the "Persuasion Attack" setting when the task images have more variations. 2. The role of the paraphrase step needs more explanations.

Questions

Overall, I enjoyed reading this paper. In particular, I found the proposed “Persuasion Attack” scenario interesting, which enables the impact of data poisoning to be more general. To my knowledge, most of the existing literature focused on targeted data poisoning (which naturally corresponds to the “Label Attack” setting for text space) or backdoor attacks, so the realization of the adversarial goal of persuasion attacks with low poisoning rates is a good contribution to the field. Nevertheless, I had the following questions related to this new threat model and the proposed attack: 1. A major difference between “Label Attack” and “Persuasion Attack” is that the targeted concepts in the persuasion scenario are much broader than the targeted phrases in the label attack. In other words, within the concepts selected by the adversary, there are many more variations of image-test pairs associated with the concepts. However, looking at Figure 10, the task images corresponding to the persuasion attack setting are not that different, which makes me wonder about the generalizability of your attack. Specifically, all the visualized images in the “JunkFood” category are hamburgers and fries, whereas the images in the “HealthyFood” category are mostly salads. Task images visualized in “VideoGame-to-PhysicalHealth” have a similar pattern. So, my questions are: (a) Will the poisoned VLM be also effective against other types of “JunkFood” (e.g., fried chickens), or the scope is essentially limited to “Hamburg and Fries”? (b) If the set of task images associated with target concepts has a larger variety in nature, how will the performance of your proposed poisoning attack change? 2. You mentioned in Section 3.3 that an LLM is employed to paraphrase the generated caption to more explicitly emphasize the concept. I do not fully understand why such a paraphrase step is necessary for your attack, especially given that you are not evaluating whether the specific “concept word” appears in the response, but instead employing another LLM as a judge to evaluate the attack success rate. The authors are highly recommended to clarify why this step is important, perhaps supported with empirical evidence on how much performance drop will be if removing this paraphrase step. 3. It would be better if the paper could present more discussions regarding the countermeasures against their attack. Looking at Figures 7-9, it seems that the attack success rates for “JunkFood-to-HealthyFood” are significantly lower than the other settings. Moreover, it seems counter-intuitive that the attack success rates further drop when the number of poisoned training samples is increased under the “JunkFood-to-HealthyFood” task. Can the authors further explain why the results are so different for this task? 4. For JPEG compression, I do not see “improved attack success rates across all tasks” when comparing the two figures in Figure 9. So, I think the authors should rewrite this sentence to avoid overclaim.

Rating

6

Confidence

4

Soundness

2

Presentation

4

Contribution

3

Limitations

As mentioned in the last section of this work, one limitation of this work is not studying defenses against VLM poisoning attacks. It would be better if the authors could add discussions on potential defense strategies that might be effective against your attacks.

Summary

This paper introduces Shadowcast, a data poisoning attack targeting Vision-Language Models (VLMs) to manipulate their responses. It features two attack types: Label Attack, which misidentifies images (e.g., confusing Donald Trump for Joe Biden), and Persuasion Attack, which generates misleading narratives (e.g., framing junk food as healthy). Effective with just 50 poison samples, Shadowcast poses a significant risk as it can manipulate VLMs across different architectures and remain potent under realistic conditions. The research emphasizes the crucial need for data integrity in VLM deployments.

Strengths

1. The topic of this paper is interesting. 2. The paper is well-organized and easy to understand. 3. The experiments conducted are comprehensive.

Weaknesses

1. The novelty of this paper is somewhat limited. The authors manipulated the original image to resemble target images in the latent space, resulting in the text caption of the original image being associated with these target images post-training [1,2]. 2. The defense mechanisms presented are inadequate. The authors should consider more robust purification methods to assess the resilience of their proposed approach, such as DiffPure [3]. 3. Certain methods could be readily adapted to this context; therefore, the authors should include comparisons with these baseline methods in their analysis, even if they are not as stealthy. [1] Carlini, Nicholas, and Andreas Terzis. "Poisoning and backdooring contrastive learning." arXiv preprint arXiv:2106.09667 (2021). [2] Shan, Shawn, et al. "Glaze: Protecting artists from style mimicry by {Text-to-Image} models." 32nd USENIX Security Symposium (USENIX Security 23). 2023. [3] Nie, Weili, et al. "Diffusion models for adversarial purification." arXiv preprint arXiv:2205.07460 (2022).

Questions

Please see the weakness.

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

Please see the weakness.

Summary

The paper introduces a subtle attack on vision-language models (VLMs). By stealthily modifying training data, the attack influences model outputs without obvious signs of tampering. Extensive experiments demonstrate the attack's effectiveness and stealthiness, revealing significant vulnerabilities in VLMs. The findings underscore the need for improved security measures to protect against such sophisticated data poisoning techniques.

Strengths

1.The paper introduces a novel clean-label attack on vision-language models (VLMs), utilizing stealthy data poisoning to manipulate multimodal outputs while maintaining a high degree of concealment. 2.The proposed method is versatile, encompassing both label attacks and persuasion attacks, demonstrating its broad applicability. 3.The approach is well-articulated, supported by comprehensive experimental evaluations, including both automated and human assessments, which validate the attack's effectiveness.

Weaknesses

1.The experiments only evaluate a Llava VLM model, one dataset, and four attack tasks. Expanding the scope to include more diverse models, datasets, and attack tasks would enhance the generalizability of the findings. 2.The impact of the PGD perturbation step size on the experimental results is not discussed. Providing a range of ablation studies would help in understanding the method's effectiveness more comprehensively. 3.The performance of ShadowCast against SOTA backdoor defenses is not explored. While the poisoned dataset is imperceptible to the human, its robustness against SOTA data anomaly detection techniques needs to be assessed.

Questions

1.How does the ShadowCast attack perform on VLMs trained on significantly larger and more diverse datasets? 2.Can the attack be adapted to target other types of multimodal models, such as those combining text with audio or other data modalities? 3.What are the implications of the attack on models deployed in high-stakes environments, such as healthcare or autonomous driving? 4.Are there specific characteristics of the poisoned data samples that make them more or less detectable by standard data validation processes?

Rating

5

Confidence

3

Soundness

3

Presentation

2

Contribution

2

Limitations

1.Scalability: How the attack scales with the size of the dataset and the complexity of the model. 2.Real-world Deployment: Potential challenges and implications of deploying the attack in real-world settings.

Summary

This paper explores the data poisoning attacks against vision language model. The Shadowcast introduces two types of attack: 1) Label Attack: the VLM generates text that misclassifies, e.g., change “Trump” to “Biden”; 2) Persuasion Attack: the VLM generates “rational” but wrong fact, e.g., illustrate the picture of “junk food” with “healthy food” description. To achieve the data poisoning, the author crafts the poisoned images inspired by clean-label poisoning; and crafts the poisoned text by replacing the text label (Label Attack) or description (Persuasion Attack).

Strengths

1.Investigating the vulnerability of VLM is an interesting and active research area. 2.Involving human evaluation is not significant, however encouraged evaluation way. 3.The paper is well-written and well-organized.

Weaknesses

1.Poisoning attacks on NLP/CV are well known. This paper is a bit incremental, seems to apply the data poisoning on VLMs. The way to craft poisoned samples are not technique significant: 1) For text, a separate VLM will first generate the image description; then the author replaces the class label (e.g., change “Trump” to “Biden”) or modifies the description (e.g., ask to describe the junk food as healthy food) by specific paraphrasing instructions. 2) For images, the authors leverage similar feature collision strategy in (Shafahi et al. [2018], Zhu et al. [2019]). In this way, the contribution of this work is trivial. 2.The author evaluates the attack effectiveness on two VQA dataset. Are there other data/tasks to indicate the attack efficiency of Shadowcast? 3.There are only limited Label/Persuasive attack examples. E.g., “Trump” to “Biden”. Are there other showcases?

Questions

1.What is the training details to get a well-trained backdoored VLM? E.g., how many epoch for two datasets? 2.I assume the data poisoning happens during instruction tuning stage. Does the author explore anything regarding data poisoning during pre-training?

Rating

5

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

N/A