Understanding and Improving Ensemble Adversarial Defense

Summary

This paper first propose to understand ensemble adversarial defense from a theoretical perspective. Then the authors develop iGAT, including a distributing strategy is to encourage each base classifier to keep improving over regions that they are relatively good at classifying. This leads to a hard distributing rule that assigning in a deterministic way the example to the base classifier that returns the highest predicted probability on its ground truth class, and a soft distributing rule based on a ranking-based score. Besides, a regularization term is introduced to address the severest weakness, by minimizing the probability score of the most incorrectly predicted class by the most erroneous base classifier. Empirical evaluations are done on enhancing previous ensemble defenses.

Strengths

This paper provides a theoretical analysis on why an ensemble of adversarially trained classifiers is more robust than single ones and develop a new error theory dedicated to understanding ensemble adversarial defense. This theoretical analysis provides some new insights on the effectiveness of ensemble defense. Inspired from the theory, the authors propose iGAT, including probabilistic distributing rule and a regularization term for rescuing the severest weaknesses of the base classifiers. Empirically, iGAT is shown to consistently enhance previous ensemble defense strategies such as ADP, CLDL, DVERGE and SoE.

Weaknesses

I have some concerns: - The designs of distributing rule in Section 5.1 and regularization in Section 5.2 are somewhat heuristic. For example, it is not well-justified on why ranking-based probability in Equation (10) is preferred compared to other soft distribution. - Does the hyperparameters \alpha and \beta have to be the same in Equation (7) and Equation (12)? If yes, please elaborate on why they have to be the same? If not, then there are too many hyperparameters (e.g., four) to tune. - In Table 1 (CIFAR-10), the empirical improvements of iGAT_DVERGE vs. DVERGE and iGAT_CLDL vs. CLDL are quite marginal under AA. - Besides, I observe that in Table 1, iGAT is combined with CLDL, DVERGE, and ADP on CIFAR-10; while iGAT is combined with SoE, ADP, DVERGE on CIFAR-100. Is there any special reason for this experimental setting?

Questions

My questions are listed in Weaknesses.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

Limitations are not discussed in the paper. As to the potential negative societal impact, there is a sentence in Conclusion.

Summary

This paper derives provable error reduction when shifting from employing one neural network to an ensemble of two neural networks, which provides new insights into ensemble adversarial defenses. Based on the theoretical findings, the authors propose iGAT to narrow the subspace where all base classifiers misclassify inputs.

Strengths

1. This paper proposes a new understanding of ensemble adversarial defenses, providing new insights into why ensemble adversarial defenses are more effective. 2. Based on the theoretical analysis, the authors propose iGAT, which aims to narrow the subspace where all base classifiers misclassify inputs by allocating global adversarial examples to appropriate base classifiers and designing a regularization term for misclassified inputs. 3. The detailed experimental results demonstrate that the proposed methods significantly improve the adversarial robustness of ensemble adversarial defenses.

Weaknesses

There is a lack of experiments with a small number of base classifiers. Will the proposed method still be effective when there are only a few base classifiers?

Questions

Why is the robustness of ensemble defenses significantly lower than the robustness of state-of-the-art non-ensemble defenses? For example, the proposed method achieves 40.36% robustness under AutoAttack on the CIFAR10 dataset with a perturbation budget of 8/255. However, state-of-the-art non-ensemble adversarial training achieves 70.69% robustness under the same conditions (See RobustBench for detail).

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The core theorem only addresses the simple case where there are only two MLP classifiers.

Summary

1. This work aims at creating a theoretical framework to explain why an ensemble of MLPs can be more robust than the single components. 2. The assumptions under which the ensemble of MLPs have more robustness are given. The first set of conditions are on each individual model itself. The second set of assumption state that the performance of each classifier is acceptable in data points that are somehow close to points of other classes. 3. Under these assumptions, Theorem 4.1 states that an ensemble of two acceptable models always performs better than the individual ones. 4. Authors propose Interactive Global Adversarial Training to boost the robustness of an ensemble.

Strengths

1. Experiments involve a wide variety of ensemble defense baselines and attacks. 2. Ablation studies are performed to test the effect of the different proposed mechanisms. 3. Assumptions in theoretical part are enumerated thoroughly. 4. Experimental setup is well described 5. The motivation to understand and enhance the adversarial robustness of ensembles is an interesting research direction. 6. Experimental results suggest that the proposed technique can indeed increase the robustness of existing ensemble training frameworks.

Weaknesses

1. Theory and practice feel a bit disconnected. Theory works for 2 models, and only MLPs, and experiments are made with 8 Resnets. 2. Some assumptions in the theoretical part could be discussed in more detail to let the reader know if the requirements and assumptions could actually hold in practice or not.

Questions

1. In Equation (5), what is the expectation over $h \in \mathcal{H}$? Are you computing the average error rate over all the family of hyothesis? 2. In Assumption 4.2 for MLPs, assumption 1 says "each activation output is i.i.d with expectation $\mu$". Is this reasonable ? What does this mean for the MLP ? Does this restrict the family of activation functions one can consider ? 3. In Assumption 4.2 for MLPs, what are the orders of magnitude of the constants $\tilde{\lambda}$ and $B$ ? How does this impact the bound on Equation (6) ? 4. Is Assumption 4.4 about "Acceptable Classifiers" always reasonable in practice ? 5. Why do you think that for CIFAR-10, the ensemble diversity framework that obtained the largest improvement from your proposed technique was ADP ? And why for CIFAR-100 it seems to be DVERGE ? 6. Why do you think that the iGAT parameters $\alpha$ and $\beta$ are so different for SoE and the rest of baselines ? 7. Which attack was used for the iGAT training added global adversarial loss? 8. How much time did training take ? Your implementation shows 2000 epochs were used, which seems huge for me. Does your method require more epochs than the baselines ? Typos:       Line 254: Theory 4.1 --> Theorem 4.1       Model size column in Table 2, Appendix 3 seems to be wrong.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

2 fair

Contribution

3 good

Limitations

Yes, they have done.

Summary

This paper investigates enhancing the performance of ensemble-based adversarial defense methods against adversarial attacks. Considering the effectiveness of ensemble-based adversarial defense methods have been proven empirically while the understanding of their mechanism is rather limited, authors first propose a theory model to demonstrate the advantages of an ensemble of adversarially trained classifiers compared to individual ones. Guided by theoretical analysis, a novel method is developed to enhance the performance of existing ensemble-based defense methods through distributing different examples to different base classifiers. Experiments conducted on various datasets verify the superiority of the proposed method.

Strengths

1. This paper proposes theoretical analysis on understanding the advantages of ensemble-based adversarial defense methods, which provides a new perspective to evaluate ensemble-based defense methods and hence could be helpful for future works 2. Based on theoretical understanding, a novel method is proposed to boost the performance of existing ensemble-based defense methods with reduced training time 3. Experiment results demonstrate the effectiveness of the proposed method

Weaknesses

1. In experiments, authors apply the proposed method to enhance four state-of-the-art ensemble adversarial defense methods. However, in Table 1, only TOP 3 most enhanced performance are provided. It would be better if the performance related to all four ensemble defense methods can be provided 2. One feature of the proposed method is distributing different examples to different base classifiers. As author mentioned, this feature can make the proposed method bring reduced training time. It would be better if some experiments about training efficiency can be conducted

Questions

1. Based on both Table 1 and Table 2, the improvement brought by the proposed method is not consistent among different ensemble-based defense methods. For example, for average combiner, the most significant performance improvement under AA is brought by the proposed method with ADP on CIFAR10 while with DVERGE on CIFAR100. Can authors provide more discussion about this phenomenon? Or, under which scenarios, the proposed method can bring most significant improvement? 2. Can authors provide the performance of all four methods in Table 1?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

If possible, authors can add some discussion about the limitations of the proposed method.

Summary

The ensemble strategy has gained popularity in the field of adversarial defense, wherein multiple base classifiers are trained cooperatively to protect against adversarial attacks. While empirical evidence supports its effectiveness, the underlying theoretical explanations for why an ensemble of adversarially trained classifiers outperforms individual classifiers remain elusive. This paper aims to bridge this gap by presenting a novel error theory dedicated to comprehending ensemble adversarial defense, providing provable reductions in the 0-1 loss on challenging example sets encountered in adversarial defense scenarios. Building upon this theory, the authors propose an innovative approach called interactive global adversarial training (iGAT) to further enhance ensemble adversarial defense. iGAT incorporates two key components: (1) a probabilistic distributing rule that selectively assigns globally challenging adversarial examples to different base classifiers, and (2) a regularization term designed to address the most severe weaknesses of the base classifiers. Evaluations conducted on CIFAR10 and CIFAR100 datasets demonstrate that iGAT significantly improves the performance of existing ensemble adversarial defense techniques. Under both white-box and black-box attacks, iGAT achieves performance gains ranging from 1% to 17%. These results highlight the efficacy of iGAT in bolstering the resilience of ensemble classifiers against adversarial attacks.

Strengths

1. Novel Error Analysis: The paper presents a new error theory specifically dedicated to understanding the advantages of ensemble adversarial defense. This analysis fills a crucial gap in the literature by providing theoretical explanations for why an ensemble of adversarially trained classifiers is more robust than individual classifiers. By offering a deeper understanding of ensemble defense mechanisms, the paper contributes to the advancement of the field. 2. Targeted Solution Algorithm: In response to the theoretical insights gained from the error analysis, the paper proposes an innovative approach known as interactive global adversarial training (iGAT). This algorithmic solution addresses the shortcomings of existing ensemble adversarial defense techniques. It introduces a probabilistic distributing rule that selectively allocates globally challenging adversarial examples to different base classifiers, along with a regularization term that targets the most severe weaknesses of these classifiers. The proposed iGAT algorithm is tailored to enhance the performance and robustness of ensemble classifiers.

Weaknesses

1. Limited Validation on Large-Scale Datasets: The paper's experimental evaluation focuses on the CIFAR10 and CIFAR100 datasets, which are relatively smaller in scale compared to datasets like ImageNet. The absence of validation on larger datasets limits the generalizability of the proposed iGAT algorithm and its effectiveness in real-world scenarios. 2. Limited Range of Attack Algorithms: The evaluation of the proposed defense technique primarily considers a limited range of attack algorithms without exploring a wider variety, including black-box attacks. While the performance improvements demonstrated against the evaluated attack algorithms are promising, it is important to assess the effectiveness of iGAT against a broader set of attack strategies to ensure its robustness and applicability in different adversarial settings. 3. Lack of Clear Explanation on the Differentiability of Hard Distributing Rule: The paper introduces a probabilistic distributing rule, referred to as the hard distributing rule, to assign adversarial examples to base classifiers. However, the paper does not provide a clear explanation regarding the differentiability of this rule. This omission raises concerns about the potential presence of obfuscated gradients, which could hinder the effective training of the ensemble classifiers and impact the overall performance of iGAT.

Questions

Please provide results on ImageNet, and explain the differentiability of Hard Distributing Rule

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

None

Thanks for the rebuttal

Thanks for providing the rebuttal. I will keep my original rating.

Thank you for your reply

I have read authors' reply, which provides a more comprehensive aspect for the proposed method, especially for the limitations. Hence, I raised my score to 7.