Towards Stable Backdoor Purification through Feature Shift Tuning

Summary

Based on the observations that the previous FT and FP methods fail at low poisoning rate scenarios, this paper finds out the potential reasons in terms of clean and backdoor feature separation degree. It proposes FST to solve the problem by disentangling the features and validates its effectiveness with multiple backdoor attacks.

Strengths

1. The paper is well-written and easy to read. No typo is found. 2. The experiments are sufficient to demonstrate the findings of previous methods’ failure in low poisoning rate scenarios. And the experiments to verify the potential reason and validate the two initial methods are also convincing. 3. The proposed methods are intuitively reasonable, which are derived step-by-step from the observations of the pre-experiments to the simple solutions, and the final version with penalty. 4. The idea is simple yet effective as shown in Sections 4 and 5, which is easy to follow.

Weaknesses

In general, I appreciate this paper, but there are still some concerns from my perspective. 1. The implementation of the proposed methods is not publicly available, only the public library ‘BackdoorBench’ is provided. 2. The effectiveness of the constraint term C in equation (1) is expected to be discussed in the ablation study.

Questions

1. Do the dynamic backdoor attacks, such as WaNet[1] and IAB[2], which generate triggers according to the input, follow the same phenomenon as in Figures 1 and 2? Since the WaNet is used in Section 5 while not in the pre-experiment part. [1] Nguyen, Anh, and Anh Tran. "Wanet--imperceptible warping-based backdoor attack." *arXiv preprint arXiv:2102.10369* (2021). [2] Nguyen, Tuan Anh, and Anh Tran. "Input-aware dynamic backdoor attack." *Advances in Neural Information Processing Systems* 33 (2020): 3454-3464.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

Please see above.

Summary

This paper studies the effectiveness of finetuning in backdoor defense with a low poisoning rate and finds that the feature entanglement at low poisoning rate affects the effectiveness of finetuning. Thus, this paper proposes 3 new finetuning strategy FE-tuning, FT-init, and FST. The experiments demonstrate a promising result.

Strengths

1. The finetuning study is interesting. 2. FST only uses finetuning to achieve the best backdoor removal results, which is very impressive. 3. This paper is well written and easy to follow.

Weaknesses

Although this work is interesting, I still have several concerns. 1. How much data is used in FST? From my point of view, the reinitialization and the larger difference loss needs more data. But if the amount of data is too large, the defenders can use these data to retrain a new model. 2. As demonstrated in the paper, only finetuning the feature extractors cannot remove backdoor successfully. If so, it demonstrates that there is poison in the linear layers, can you explain this phenomenon? 3. Can you provide more TSNE results against more attacks like Figure3 such as BadNets and WaNet? 4. Although this paper focuses on a low poisoning rate, I think results on poisoning rate such as 20% should be also considered. Because the defenders cannot know the real poisoning rate and the robustness of different poisoning rates is important.

Questions

1. How much data is used in FST? From my point of view, the reinitialization and the larger difference loss needs more data. But if the amount of data is too large, the defenders can use these data to retrain a new model.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

4 excellent

Limitations

Yes

Summary

This paper observes that while fine-tuning and linear probing can act as effective defenses in the high-poisoning-rate regime, they completely fail in the low-poisoning-rate regime. The paper further shows that this is due to the fact that in the low-poisoning-rate regime, extracted features of backdoored and clean samples are highly entangled. The proposed defense is to fine-tune the backdoor model with an additional constraint to force the fine-tuned weights to differ from the original (poisoned) weights. Extensive experiments show that the proposed defense provides excellent backdoor purification while maintaining clean accuracy.

Strengths

- This paper shows an interesting observation that fine-tuning and linear probing perform significantly worse as backdoor defenses in the low poisoning rate regime. - This observation is followed by a thorough and nicely written analysis in Section 3. - The proposed defense is simple, elegant, yet effective. - Very extensive experiments.

Weaknesses

Please pay more attention to weaknesses marked with "major" severity. - **[minor, clarity]**: L52, please also mention that FE-tuning randomly re-initializes the classifier head. I was confused when looking at Figure 1 at first because it was not mentioned previously that FE-tuning does re-initialize $f(w)$. - **[minor, typo]**: L102, "Here We" -> "Here, we" - **[major, lack of experiments on high poisoning rates]**: The proposed defense should be tested with high poisoning rates just to ensure that it performs equally well.

Questions

- It would be nice if the authors could extend Figure 2 to other datasets. Specifically, I am interested in seeing the results on Tiny-ImageNet and CIFAR-100.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

- One potential limitation of this work is the assumption of having access to clean data for training, which may not always be practical in real-world scenarios. However, the authors have acknowledged this limitation and expressed their commitment to addressing it in future research. - It would have been more insightful if the authors had delved deeper into why low poisoning rates cause entanglement between backdoored and clean features.

Summary

This paper finds that fine-tuning is less effective in defending against backdoor attacks with a low poisoning rate, due to the strong coupling between the clean feature and the backdoor feature. Therefore, a tuning-based backdoor purification method called feature shift tuning (FST) is proposed, which is simple and efficient. FST can effectively decouple the clean feature and the backdoor feature, and thus eliminate the backdoor in the victim model.

Strengths

1. This paper proposes a simple and effective backdoor elimination method, which achieves good results in a specific scenario where the poisoning rate of the backdoor attack is relatively low.

Weaknesses

1. This paper finds that fine-tuning has difficulty defending against backdoor attacks with a low poisoning rate when ResNet-18 is the victim model, but does not sufficiently demonstrate the generalizability of the problem. The same claim is not guaranteed to hold when the model capacity, model architecture, and dataset type change. 2. This paper lacks an explanation of why the method works and does not explore the effect of the range of poisoning rates on the victim model. 3. It is desirable to include backdoor elimination methods other than tuning-based methods as baseline methods as well to demonstrate the efficiency of FST.

Questions

Please refer to weakness.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

It is suggested that the authors provide further explanation as to why the method worked.

Summary

This paper proposes a novel backdoor defense approach called Feature Shift Tuning (FST) that actively promotes feature shifts to disentangle the learned features of a deep neural network. The authors conduct a thorough assessment of widely used tuning methods, vanilla Fine-tuning (FT), and simple Linear Probing (LP), and demonstrate that they both completely fail to defend against backdoor attacks with low poisoning rates. FST is an end-to-end tuning method that actively shifts features during fine-tuning by encouraging the difference between the tuned classifier weight w and the original backdoored classifier weight w_ori. FST significantly shifts backdoor features and makes them easily separable from clean features of the target class, which effectively mitigates backdoor attacks. In summary, the proposed FST method is a strong backdoor defense paradigm that can achieve stable improvements in both robustness and clean accuracy compared to other initial methods such as FE-tuning and FT-init.

Strengths

- The proposed FST (Fine-tuning with Separated Transformations) method is a strong backdoor defense paradigm that can achieve stable improvements in both robustness and clean accuracy compared to other initial methods such as FE-tuning and FT-init. FST is an end-to-end tuning method that actively shifts features during fine-tuning by encouraging the difference between the tuned classifier weight w and the original backdoored classifier weight w_ori. This is giving a good reference to the research on this aspect. - FST disentangles the clean and backdoor features, which helps to purify the model. It significantly shifts backdoor features and makes them easily separable from clean features of the target class, which effectively mitigates backdoor attacks. FST outperforms other methods by a large margin for backdoor robustness on CIFAR-10 and GTSRB datasets. - FST is effective against adaptive attacks such as the Adaptive-Blend attack, achieving an average drop on ASR by 81.66% and 91.75% respectively for the CIFAR-10 and GTSRB datasets. FST is a unified method that addresses both the clean accuracy drop and unsatisfied robustness improvements that are present in other backdoor defense methods. The ablation study is organized well to clearly demonstrate the whole proposed method. And it makes the paper easy to follow.

Weaknesses

- [Contribution and novelty] The contribution of paper is somehow incremental, as the method is developed based on previously researched findings. The backdoor attack can be erased by decoupling the feature extractor and the linear classifier in [1]. - [Adaptive Attack] There, the submission only briefly mentions an attempt at achievingthe disentanglement between the clean and backdoor features through an adaptive attack, but fails to do so effectively, and provides only limited insight why the adaptive attack failed. Please take a closer look at providing a strong evaluation of the adaptive attack scenario. Besides, the paper [2] also proposes another defense, Adaptive-Patch, which should also be discussed when attacking the propsoed FST. [1] Huang, Kunzhe, et al. "Backdoor Defense via Decoupling the Training Process." International Conference on Learning Representations. 2021. [2] Xiangyu Qi, Tinghao Xie, Yiming Li, Saeed Mahloujifar, and Prateek Mittal. Revisiting the assump405 tion of latent separability for backdoor defenses. International Conference on Learning Representations. 2023.

Questions

Listed in the weakness of the paper. Score can be improved if concerns listed above are resolved.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors addressed the limitation that they assume the defender would hold a clean tuning set which may not be feasible in certain scenarios.

All concerns addressed

Thank you for your response. Please consider adding your intuition for Limitation 2 in the revision. I am raising my rating to 7.

Thanks!

We appreciate your further response and your recognition of our work. We will include our discussion in the revised version. The authors

Thanks for your recognition of our work

We sincerely appreciate your constructive feedback throughout the review process. We are committed to incorporating your suggestions as we revise the paper. We are delighted that our responses have addressed your concerns. Thanks for your recognition of our work! Best regards, The Authors