Unveiling and Mitigating Backdoor Vulnerabilities based on Unlearning Weight Changes and Backdoor Activeness

Summary

This paper first makes two observations that neurons exhibiting significant weight changes during clean unlearning also tend to play crucial roles in poison unlearning, and neurons in the backdoored model are always more active compared to those in the clean model. The authors showcase on commonly used backdoor attacks and provide further explanations for these two observations. Based on these two observations, a model is first unlearned on clean data where weights with the highest changes are re-initialized, and then the model is optimized by activeness-aware fine-tuning. Extensive experiments are provided to support the proposed defense.

Strengths

The analysis is clear and well-formulated, based on which the proposed defense is effective and easy to interpret. Extensive experiments on different types of backdoor attacks are provided.

Weaknesses

The design principle of the proposed method (TSBD) is very similar to RNP [22]. Both TSBD and RNP follow the two-stage defense setup, including clean unlearning and recovering. The performance of TSBD is also similar to RNP except for the SIG backdoor. The authors provide some explanations about the difference between RNP and TSBD, but they are not convincing. The core technical difference can be further clarified. Table 3 provides the ablation study on the zero reinitialization ratios, indicating that reinitialization is a sensitive parameter. In a practical threat scenario, how would the defender select a proper reinitialization ratio? It is also suspicious whether the neuron ratio selection plays a key role in increasing the effectiveness. Closely related work [a] is missing, where clean unlearning by mask optimization is also discussed. [a] Towards reliable and efficient backdoor trigger inversion via decoupling benign features. In ICLR, 2024

Questions

My main concern is the difference between TSBD and RNP, as well as the practicality of TSBD as a defense.

Rating

5

Confidence

3

Soundness

2

Presentation

3

Contribution

3

Limitations

The authors did not provide analysis on limitations of the proposed method.

Summary

The authors propose a novel two-stage backdoor defense method TSBD. The proposed method is based on two key observations 1) the weight changes of neurons during clean and poison unlearning are correlated, 2) the backdoored neurons exhibit a larger gradient norm during unlearning. Respectively, the proposed defense method consists of 1) reinitializing neurons with high weight changes, 2) fine-tuning under gradient-norm regularization. The proposed method is compared with several state-of-the-art defense methods on different attacks, and a comprehensive ablation study is conducted.

Strengths

1. The paper is well-written and easy to follow. The figures are very clear. 2. The observations are interesting, providing insights into this field. 3. The proposed method is effective across different attacks, showing low attack successful rate and good defense effective rate. The authors provide sufficient and convincing results.

Weaknesses

1. How is the performance on scaled-up experiments, e.g. ViT, or ImageNet? 2. What is the computational overhead of the proposed method compared to others?

Questions

Elaborated above.

Rating

8

Confidence

3

Soundness

4

Presentation

4

Contribution

4

Limitations

The authors discuss the limitations.

Summary

The paper addresses the security threat posed by backdoor attacks in deep neural networks (DNNs). The authors explore model unlearning from the perspective of weight changes and gradient norms, making two key observations: weight changes between poison and clean unlearning are positively correlated, and neurons in backdoored models are more active than those in clean models. Based on these observations, they propose a Two-Stage Backdoor Defense (TSBD) method, involving Neuron Weight Change-based Backdoor Reinitialization and Activeness-Aware Fine-Tuning. Extensive experiments demonstrate the superior performance of their method compared to state-of-the-art approaches.

Strengths

1. The paper introduces a novel perspective on backdoor defense by exploring the correlation between weight changes in poison and clean unlearning and the activeness of neurons. This approach provides new insights into identifying and mitigating backdoor vulnerabilities. 2. The paper is well-organized and clearly presents its methodology, findings, and contributions. 3. The proposed TSBD method is rigorously evaluated through extensive experiments involving eight backdoor attacks on three benchmark datasets.

Weaknesses

1. The TSBD method involves additional steps such as clean unlearning, neuron weight change calculation, and activeness-aware fine-tuning, which may introduce computational overhead. An analysis of the computational cost and efficiency of the proposed method compared to existing defenses would be beneficial. 2. What causes the clean unlearning NWCs to exhibit a positive correlation with those in poison unlearning? I think this question has not been solved clearly.

Questions

1. When selecting the clean dataset, how is the class distribution handled? Is it randomly selected or fully covered? Intuitively, would data related of the target label cause particularly high activation values? 2. I think the NWC strategy is not significantly different from the pruning method in RNP[1], and the Activeness-Aware Fine-tuning is also similar to the fine-tuning in RNP. Therefore, I hope the authors can emphasize more clearly the differences between their approach and RNP. 3. From the observations, it is apparent that there is a difference in neuron activation between clean and backdoored models. However, the use of gradient norm restriction in the unlearning process is applied to an already modified model (not the original backdoored model). The direct correlation between this and enhanced defense effectiveness does not seem sufficiently clear and is not within the scope of the initial observations. 4. Regarding the selection of hyperparameters in Activeness-Aware Fine-tuning, I believe this can significantly impact the experimental results, but there is no discussion on this aspect in the paper. [1] Li Y, Lyu X, Ma X, et al. Reconstructive neuron pruning for backdoor defense[C]//International Conference on Machine Learning. PMLR, 2023: 19837-19854.

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

I think the paper still has some limitations that the authors didn't mention. Please refer to my questions and weaknesses. Please add some discussions about data-free methods

Summary

The paper introduces two key observations in backdoored models, presenting a two-stage backdoor attack defense method, where the two observations are the following: a strong positive correlation between weight changes in poison and clean unlearning, and the stronger neuron activation in backdoored models compared to clean models. The proposed two-stage defense method leverages these observations by reinitializing a certain proportion of backdoor-related neurons and subweights, and by suppressing gradient normalization during the fine-tuning process. The proposed method demonstrates the state-of-the-art performance on the selected datasets and models with a wide range of ablation studies.

Strengths

1. The paper introduces a novel approach to backdoor defense. It provides two insights in the weights of backdoor models. Based on this, the authors introduce a two-stage defense mechanism. Also, the use of clean unlearning that does not require poisoned data to prevent the backdoor attack. 2. While the scope of the experiment is limited in model architectures and datasets, the experiment shows superior performance. The experimental setup is thorough, covering a certain range of backdoor attacks. 3. The paper is well-organized, with figures and tables are used to illustrate key observations and experimental results. The descriptions of the observations and the proposed defense method are detailed and precise.

Weaknesses

1. The scale of experiments are too limited. While the authors compare the results between 8 other backdoor attack methods, the main experiments are only conducted to specific datasets and models (CIFAR-10, PreAct-ResNet18). This makes the generalizability of the results rather unclear. 2. The authors mention computational efficiency in Activeness-Aware Fine-Tuning by using an approximation scheme, but there are no experiments demonstrating the trade-off between efficiency and accuracy. Including such experiments would provide a more comprehensive understanding of the method's practical implications. 3. The concept of unlearning in backdoor attacks could be better explained in the Related Works section. Given its importance in this paper, a more detailed discussion would help contextualize the proposed method and highlight its significance in addressing existing challenges. 4. There is no mathematical proof. While it employs various loss functions and approximation techniques, and the empirical results are quite promising, there is no analysis, explaining why the proposed method works.

Questions

1. There is no statement or analysis about scalability and generalizability. Can you provide more detailed information about these, to different datasets and model architectures? 2. How can the clean unlearning process identifies and affects backdoor related neurons, without causing degradation of the models’ performance on clean data? 3. The section introducing the effectiveness of zero reinitialization on subweights in 4.3 Ablation Studies is a bit confusing. The difference between V2 and V3 is still not clear to me. 4. I have questions about the insensitivity of TSBD to both neuron ratio and weight ratio as explained in Section 4.4. While I understand that the method effectively selects backdoor-related neurons indicated as active neurons and that reinitializing them is effective, it is confusing that the performance remains similar across both low and high ratios. Is reinitializing itself the important part?

Rating

4

Confidence

3

Soundness

2

Presentation

2

Contribution

3

Limitations

The authors mention in the introduction that clean unlearning is still underexplored, but it is unclear if this limitation has been fully addressed in the paper. Additionally, the paper lacks a discussion of its own limitations. Providing a clear discussion on the limitations of the proposed method and potential areas for future research would enhance the paper. The authors acknowledge that there remain challenges in fully mitigating backdoor effects without any access to clean data. They suggest that data generation techniques and data free techniques might offer potential solutions. There is no discussion about the scalability of their proposed method, especially in large scale or real time applications. Addressing the computational overhead and resource requirements would be necessary for practical applications.

Concerns about the generability of proposed shceme

Thanks for considering my concerns and I see the authors have provided more evaluations for validating its feasibility and effectiveness. An additional minor question is that How compatible is this NWC-based Backdoor Reinitialization approach to "DHBE: Data-free Holistic Backdoor Erasing in Deep Neural Networks via Restricted Adversarial Distillation", AsiaCCS and "SupRTE: Suppressing Backdoor Injection in Federated Learning via Robust Trust Evaluation". Besides, it is already difficult to innovate in this direction, and the limitations of NWC-based Backdoor Reinitialization methods should be discussed.