Label Poisoning is All You Need

Summary

This work introduces a novel backdoor attack, (soft)FLIP, which only requires modifying the labels of the training samples and supports arbitrary choice of triggers. (soft)FLIP is based on a trajectory-matching algorithm, which optimizes the poison labels by simulating how normal backdoor poison samples affect the model parameters during training. The experimental results validate the high effectiveness of (soft)FLIP on CIFAR10 and CIFAR100, across two ResNet architectures, three different types of triggers, and different numbers of poison samples.

Strengths

1. The idea of backdoor poisoning only modifying the label is rather novel and interesting. Meanwhile, the reported attack results look quite promising. 2. The motivation of label-only poisoning backdoor attack is well supported by two real-world scenarios (crowd-sourcing image annotation & knowledge distillation). Overall, the paper is well structured. 3. The time consumption to conduct FLIP could go less than 1 GPU hour, making the proposed attack more practical.

Weaknesses

1. My major concern is about the limitation of the model architectures you investigated. Through both the main paper and the appendix, you only investigate ResNet architectures (r18 & r32), which only cover a small spectrum of CV deep models. The lack of model architecture diversity subsequently leads to readers' doubt --- whether your attack could generalize well in the real world, on more recent architectures which are not in the ResNet family. I think for the paper's acceptance, it's necessary to consider other types of model architectures (e.g. VGG, Vision Transformer). 2. Another concern I have is the lack of studies from the defender's perspective. On the one hand, there are many existing advanced poison-cleanser defenses (e.g. [1] and [2]). I suggest including a brief study about whether your poisoning attack is resistant to these defenses in your paper's main body. It would be even better if you provide some insights into potential defenses that may effectively resist FLIP. 3. The logic in paragraph 2 (Line 30-36) may need refining (e.g. "However" in Line 32 is inappropriate). 4. Line 100-101 the claim "the corrupted labels chosen by FLIP do not degrade the clean accuracy of the model" should be diminished, since FLIP still leads to some CTA drop. 5. The caption for Figure 5(a) is wrong, please correct it. 6. May need to add identifiers (e.g. number of experts & number of poisoning samples) in Table 4. I will update my score once the above concerns are addressed. [1] Hayase, Jonathan, Weihao Kong, Raghav Somani, and Sewoong Oh. "Spectre: Defending against backdoor attacks using robust statistics." In *International Conference on Machine Learning*, pp. 4129-4139. PMLR, 2021. [2] Tang, Di, XiaoFeng Wang, Haixu Tang, and Kehuan Zhang. "Demon in the variant: Statistical analysis of {DNNs} for robust backdoor contamination detection." In *30th USENIX Security Symposium (USENIX Security 21)*, pp. 1541-1558. 2021.

Questions

1. Your attack setting requires selecting both a (coarse) source class and a target class. What about all-to-one attack (every class is the source class, which is more common and more practical)? 2. In Line 195-198, what if $C$ is small? Could you explain more about the selection and meaning of $C$? 3. Could you provide results using all three trigger styles, for CIFAR-10 (r18) and CIFAR-100 (r18 & r32) settings? 4. There are some recent backdoor poisoning attacks (e.g. [1] and [2]) that also emphasize the manipulation of poison labeling. What's the relationship between your work and them? It would be great to include a short discussion in Related Work. [1] Tang, Di, XiaoFeng Wang, Haixu Tang, and Kehuan Zhang. "Demon in the variant: Statistical analysis of {DNNs} for robust backdoor contamination detection." In *30th USENIX Security Symposium (USENIX Security 21)*, pp. 1541-1558. 2021. [2] Qi, Xiangyu, Tinghao Xie, Yiming Li, Saeed Mahloujifar, and Prateek Mittal. "Revisiting the assumption of latent separability for backdoor defenses." In *The eleventh international conference on learning representations*. 2022.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Limitations and potential defenses against the proposed attack are not explicitly discussed.

Summary

The paper studies backdoor attacks, in which a subset of training examples are poisoned with the goal of flipping predicted labels at test time with minimal modifications to the test instances (e.g., by adding a small logo or invisible noise). Previous backdoor attacks either inject malicious data or modify the instances, usually by adding a pattern that correlates with the preferred output label. This work shows that by *only changing the labels* of a small subset of the training examples one can still launch a successful backdoor attack. At a technical level, the attack first performs "traditional" attacks to train multiple poisoned (backdoored) models while storing certain information about such attacks regarding their trajectory of the poisoned models. Then such trajectories are used to find out a minimal set of labels whose flipping can have a close effect (to the original attacks). Concretely, by only changing 1000 example labels in CIFAR-10, the paper can achieve accuracy under attack (i.e., successfully flipping the label to the desired one) of ~1 while the actual accuracy without attack remains ~0.9 The paper compares their (in my opinion impressive) attack with a natural (but rather weak) attack that is based on inner product, in which the labels are flipped if the trigger patter is "weakly" present in an input instance (and that is estimated using inner product between the instance and the pattern). The comparison shows that the proposed attack is significantly more effective.

Strengths

The main question of the paper is a natural question, and the answer is rather surprising. I personally would not think that label change is so powerful to launch a backdoor attack. The attack's idea is smart and could find further applications in adversarial learning by reducing attack power to label-only setting.

Weaknesses

The main weakness of the paper is its poor presentation, in which many sentences and phrases are unclear for readers. Here are some examples. * calling a dataset with flipped labels "clean data" (in line 105) is a misnomer. you can call them "clean instances" but the data is poisoned, as labels are also part of the data. * I did not understand the exact definition of the attack in the knowledge distillation setting. Please give a formal mathematical definition. There are many unclear sentences and phrases such as: "we convert our continuous logits to a 170 discrete set of label flips." "a set of training trajectories of backdoored ‘expert models" "Intuitively, each gradient step differs only by the poisoned images" Letter T is overloaded: it is used both for the trigger and the length of the saved trajectories. What does "intensity and stealth" of the attack mean, formally? What is "interpolation percentage"?

Questions

Refer to the weakness comments and explain the phrases and formally define the attack model for the knowledge distillation setting. Also, what is the (time) complexity of your attack. Can you report a few numbers?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

The computational complexity of the attack is not clear to me. it seems like your attack is much heavier than the "traditional" ones. but since this is an attack (not a defense) the complexity is still more OK, as there is no real symmetry between the two.

Summary

This paper studies an interesting problem whether backdoor attacks are effective if only manipulating the labels. They propose a method based on searching labels that can match parameters with pre-trained conventional backdoored models. An effective algorithm is shown and experiments are conducted to validate the effectiveness.

Strengths

The problem itself is important and the motivation is proper. The logic of the whole paper is clear and easy to follow. The algorithm is clearly illustrated, and experiments are conducted to show the effectiveness of the proposed method.

Weaknesses

1. Overall, this paper seems to be written in a rush as many details are missing. 2. Scenario two needs more demonstration. It is not clear which model(teacher or student) authors would like to poison. 3. It is not sure whether this work is the first study of label-poison backdoor, as *Clean-image Backdoor: Attacking Multi-label Models with Poisoned Labels Only* already studied a similar problem. A comparison is needed in related work, and it should be considered as a baseline. 4. Proposed method is not a novel one, because in the paper 'Stronger data poisoning attacks break data sanitization defenses', they proposed to search for poisoning samples based on a decoy model. From my perspective, the method proposed in this paper is a direct application of that method to the backdoor attack. 5. The guarantee or intuition behind the proposed matching procedure(iterations from line 212) is not clear. It is not straightforward to understand why updating poisoned labels at each checkpoint will match the parameters between expert models and label-poisoned models, because the re-training process can lead to totally different local minima. 6. Please give more details about the dot-product baseline. If it is proposed in previous works, please provide references. If it is defined in this work, show more details especially the difference between it and the proposed method.

Questions

1. I am confused about the attackers' ability in this work. Are you assuming that the clean training set is also accessible to attackers? Otherwise one can not train expert models using conventional backdoor methods. Since you need to train expert models, do you assume that the images can be manipulated? If so, is this assumption too strong in practice and does it contradict the motivation? 2. Need more details about training expert models. Are you training multiple expert models using the same architecture, trigger, source class and target class? If so, what is the difference between these expert models? Besides, expert models are only trained for 20 epochs, why not train for more epochs? 3. What is the intuition behind the loss function in Eq.(2)? Give more details about why it is designed in that form. 4. Why lack the experiments facing backdoor defenses?

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

Mentioned in weakness and questions.

Summary

The paper first proposes a method that a backdoor attack can be done with only label poisoning. The authors introduce a new algorithm called FLIP that corrupts only the labels in the training set to create a backdoor attack. The first step of FLIP is to collect a set of training trajectories of backdoored ‘expert models’. Then, a set of real-valued logits are produced that induce similar parameters to the poisoned data when combined with clean images. Moreover, parameter-matching loss are proposed to update the logits. Finally, label flips are implanted by logits. What’s more, SoftFLIP is applied in the setting of knowledge distillation. The method is evaluated on extensive experiments and shows its effectiveness.

Strengths

The article reveals the threats of only label poisoning which is rarely considered before. The method proposed seems correct and the experiments are sufficient. The experiments’ settings are rational and the article is well organized.

Weaknesses

There are no any experiments on against backdoor defense. It is also an important aspect of evaluating the backdoor attack. The backbones are limitated to the ResNet. How about the results on transformers which are more popular backbones in knowledge distillation?

Questions

In Table 1, r18 s has a poor condition on PTA, it seems strange. Do your method have the robustness against fine-tuning?

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

3 good

Contribution

4 excellent

Limitations

No any backdoor defenses are tested in the article.

Summary

This manuscript proposes a new backdoor method for two typical scenarios: crowd source annotation and knowledge distillation. The adversaries are assumed to control the training dataset and inject a backdoor through label poisoning. Specifically, the method train several backdoored models using data poisoning methods. Then, it optimize continuous logits to align clean samples and poisoned ones. Finally, it convert the continuous logits to discrete labels. Experimental results on two datasets and two resnet models demonstrate the effectiveness of the proposed mothod.

Strengths

1. The idea of poisoning labels instead of samples is interesting and novel. 2. The manuscript has conducted several exeperiments to evaluate the proposed method.

Weaknesses

1. The two scenarios are common in backdoor learning. Data posioning methods can also be exploited in such scenarios. 2. FLIP needs to train expert models first, where data poisoning models are trained as experts. It would be better to explain why not use data poisoning directly. The necessity of applying FLIP is not convincing. 3. Though FLIP is a new backdoor scheme, from my perspective, it should be compared with data poisoning backdoor models. It's hard to evaluate the performance of FLIP. In other words, FLIP needs to train data poisoning models first, what are the advantages and gains of FLIP compare its first step? 4. Larger models and datasets should be used to evaluate the models.

Questions

1. The advantages of FLIP over previous backdoor methods should be clarified from both motivation and experiment aspects. 2. How much additional computation caused/overhead caused by FLIP compared to a single data poisoning model? 3. Why not conduct experiments on larger datasets and models?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

1. The authors have not mentioned the limitations of this work. It is worth noting that the benefits of label poisoning backdoor method is not clear. The two motivation scenarios can be also attacked by existing data poisoning methods. 2. There could be potential negative societal impact of this work, if the proposed method is utilized by malicious attackers.

Thanks for the clarifications

I will follow up if I have more questions. For now I do not have more.

Thanks for the response

Thanks for the clarification. The repsonse has addressed my concerns. The scenarios of label poisoning are interesting. I would like to change my score to 5. Please clarify the threat model in the paper and add the additional results to the paper.

Thanks for your efforts in the rebuttal

I appreciate the efforts the authors made during the rebuttal period, which has addressed most of my concerns. I will increase my rating to 5 for now. It would be great if in the following days, you could also 1) show some other possible experiment settings (e.g. vit -> vit, vgg -> vit) in "Experiments on Larger Models + Transformers." (even if they fail); 2) provide results regarding my 3rd concern in "Questions" about all three triggers.