Uncovering, Explaining, and Mitigating the Superficial Safety of Backdoor Defense

Summary

This paper studies post robustness of backdoor purification. The authors show that backdoors purified by existing defenses can be recovered via Retuning Attacks and they propose the Query-based Reactivation Attack to recover the backdoor. The authors address such an vulnerability by proposing a Path-Aware Minimization defense. Evaluations are performed on four different attacks over CIFAR-10/100 and Tiny-ImageNet across various deep neural networks. Results show that the proposed PAM defense achieves a better post purification robustness.

Strengths

1. It is novel to study the post purification robustness and this paper reveals an often-neglected vulnerability of DNNs and limitations of existing defenses. 2. The paper progressively demonstrate that models can relearn backdoors after purification via Retuning Attacks and based on the observation, the authors propose a Query based attacks that is more practical under real-world threat models. 3. The finding that the inadequate deviation of purified models from the backdoored model along the backdoor-connected path is the root cause of poor post purification robustness is both Instructive and insightful. The authors successfully develop an effective defense upon this finding.

Weaknesses

1. My primary concern is that the attacks evaluated in the paper are not SOTA. BadNets, Blended, SSBA and LC are attacks developed years ago. The authors should also provide the results on recent attacks such as Sleeper Agent[1] and Adaptive Blended[2]. 2. The authors mentioned there is a trade-off between post purification robustness and the clean accuracy. However, it is unclear how to determine the hyper parameter $\rho$ to achieve the balance. An algorithm for $\rho$ selection would be necessary. 3. The presentation needs to be improved. For example, order of Fig.2 and Fig.3 should be interchanged; it is very hard to see the results in Fig. 5; Algorithm 1 should be placed close to where it is described. 4. Some typos. For example, line 142: we following -> we follow; in Figure 1, shouldn't P-ASR be R-ASR? [1]. Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch, NeurIPS, 2022 [2]. REVISITING THE ASSUMPTION OF LATENT SEPARABILITY FOR BACKDOOR DEFENSES, ICLR 2023

Questions

1. In Table 2, the clean accuracy of all these models are lower than usual, why is that?

Rating

6

Confidence

5

Soundness

3

Presentation

2

Contribution

3

Limitations

Limitations has been addressed in "Conclusions and Limitations".

Summary

This paper investigates the effectiveness of current purification-based backdoor defenses and tries to uncover whether purified DNNs are truly free from backdoor vulnerabilities. The authors identify the “post-purification robustness” of DNNs and propose Retuning Attack (RA) and Query-based Reactivation Attack (QRA) respectively to assess the susceptibility of purified DNNs to backdoor reactivation. Additionally, the paper proposes Path-Aware Minimization (PAM) to improve post-purification robustness.

Strengths

The work makes contribution in backdoor robustness evaluation by shifting the focus from merely achieving low ASR to evaluating the post-purification robustness of backdoor defenses. The paper introduces a pipeline of methods: RA, QRA, and PAM, for the assessment of this vulnerability.

Weaknesses

The proposed notion of post-purification robustness lies with the assumption that the purified DNN will encounter further fine-tuning. As such, the entire workflow proposed is not practical in a real-world scenario where the purified model is kept frozen with no further updates.

Questions

Can the authors further justify the proposed threat model’s real-world implications? How can the proposed workflow be enabled in a real-world attack setting?

Rating

6

Confidence

4

Soundness

2

Presentation

3

Contribution

4

Limitations

The paper falls short in addressing practical implementation challenges. These limitations suggest that the proposed methods might not be as universally practical as claimed.

Summary

This paper reveals a phenomenon in backdoor defense: the purified backdoor can be reactivated by fast retuning on a few backdoor samples. Building upon this observation, the paper explores both attacks and defense measures for more reliable backdoor research. On the attack side, a Retuning Attack (RA) is proposed and generalized to the black-box setting as a Query-based Reactivation Attack (QRA). On the defense side, a Path-Aware Minimization (PAM) method is proposed to force more deviation from the backdoor-connected path. Experiments verify the effectiveness of PAM compared to exact purification (EP).

Strengths

1. The observed phenomenon that the purified model is more easily to restore the trigger is interesting. 2. Both attacks and defense were explored. 3. The defense results look promising.

Weaknesses

1. The restoration of the backdoor was conducted on a poisoned subset which is not a surprise in this case as the model will surely relearn the backdoor. The authors should prove the phenomenon via tuning on purely clean training data. 2. The threat models of the two proposed attacks are problematic. 1) RA will require a post-purification poisoning to the defense model, which means that it can poison again after the defense, which does not make sense to me; 2) QRA attack is very similar to an adversarial attack, that requires access to both the purified model and the RA model, why the defender exposes these models (or their APIs) to the attacker is questionable. 3. The authors should clearly define what is Post-purification Robustness. In the current version, it appears the same as standard backdoor robustness, i.e., how to guarantee the PAM purified model is 100% robust? 4. The authors seem to confuse a backdoor attack with adversarial perturbation in proposing the QRA attack, A backdoor attack does not make changes to the input during the inference stage otherwise it becomes an adv attack. These two attack types assume different capabilities and flexibility of the adversary. 5. The proposed PAM is very much like using a moving average to force larger updates of the weights. PAM requires reverse-engineered backdoor samples $\mathcal{D_r}$, I wonder if one could simply use unlearning to achieve the same effect. 6. The proposed method was not compared with existing defense methods.

Questions

1. Can PAM deal with multi-trigger attacks?[1] [1] Li, Yige, et al. "Multi-Trigger Backdoor Attacks: More Triggers, More Threats." arXiv preprint arXiv:2401.15295 (2024). 2. How does PAM work, compared to applying the same existing defense twice, e.g., ANP or SAM? 3. Can the problem be addressed by simply adjusting the hyperparameters of the Mode Connectivity defense [2]? [2] Zhao, Pu, et al. "Bridging mode connectivity in loss landscapes and adversarial robustness." arXiv preprint arXiv:2005.00060 (2020).

Rating

5

Confidence

4

Soundness

2

Presentation

2

Contribution

2

Limitations

1. Problematic threat model. 2. Limited technical novelty. 3. Missing systematic comparison with existing methods.

Summary

Backdoor attacks are a major threat to Deep Neural Networks (DNNs), as they allow attackers to manipulate model predictions with backdoor triggers. Existing purification methods reduce the Attack Success Rate (ASR) of these models, but it's unclear if they fully eliminate backdoor threats. This study investigates post-purification robustness by employing the Retuning Attack (RA) and finds that current methods are vulnerable, as models quickly relearn backdoor behaviors. To address this, the study proposes the Query-based Reactivation Attack (QRA) and a novel Path-Aware Minimization (PAM) technique. PAM enhances robustness by promoting deviation along backdoor-connected paths with extra model updates. Extensive experiments show PAM significantly improves robustness, maintaining low ASR and good accuracy, providing a new perspective on evaluating and improving backdoor defenses.

Strengths

1. The paper empirically verifies that poisoned nodes typically exhibit large prediction variance under edge dropping, providing a indicator for identifying poisoned nodes​​. 2. The proposed robust training strategy not only has theoretical guarantees but also shows practical effectiveness in defending against various types of backdoor attacks, maintaining clean accuracy while reducing the attack success rate​​.

Weaknesses

1. While the paper focuses on various backdoor attack types, the scope of attack types and defense mechanisms explored could be broadened to cover more diverse scenarios and setting

Questions

This is a good paper, discussing the details of the proposed defense method, one thing I'm curious is that does the author has plans for designing an adaptive attack for this defense method, and how to achieve it?

Rating

7

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

The limitations have been discussed in the paper.

Thanks for your recognition of our work

We sincerely appreciate your constructive feedback throughout the review process and will incorporate your suggestions as we revise the paper. We are delighted that our responses have addressed your concerns. Thanks for your recognition of our work! The Authors.

Seeking Your Valuable Feedback

Dear Reviewer YFJE, We wish to express our gratitude for your dedicated time and insightful comments. We are awaiting your valuable feedback and insights regarding the points we addressed in the rebuttal. Ensuring your satisfaction with our rebuttal is of utmost importance to us. Your response is very helpful in further improving the quality of our work. Sincerely, Authors