Perturbation Towards Easy Samples Improves Targeted Adversarial Transferability

Summary

The paper theoretically and experimentally demonstrated that adding perturbations towards High-Sample-Density-Regions (HSDR) of the target class can further enhance the transferability of targeted attacks. To avoided the impracticality of density estimation in high-dimensional data, authors propose a method for identifying samples within HSDR. The experimental results demonstrate the effectivenss of the proposed method.

Strengths

- The experiments demonstrate the effectiveness of the proposed ESMA, which can outperform baselines by a large margin. - The idea of adding perturbations towards HSDR is intersting and effective. - To avoided the impracticality of density estimation in high-dimensional data, this paper propose a method for identifying samples within HSDR, which is practical.

Weaknesses

- The recent transfer-based attack [1,2,3] is missing. I suggest authors discuss or compare propsed method with these paper. - The experiments are not extensive, i.e., only two tables. More ablation studies should be presented. For example, - Will the results be really poor if a difficult sample is selected to form target anchor sets? Or why author don't report the results of ESMA in Table 1. - Does ESMA still outperform the existing algorithm if the source model is an ensemble of models. [1] Zhang et al. Staircase sign method for boosting adversarial attacks, 2021 [2] Zhao et al. On Success and Simplicity: A Second Look at Transferable Targeted Attacks, NeurIPS 2021 [3] Long et al. Frequency Domain Model Augmentation for Adversarial Attack, ECCV2022

Questions

Please refer to weakness.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

This paper only considers CNNs and it is doubtful whether it is valid on ViTs.

Summary

This paper presents a hypothesis that pushing adversarial samples towards high-density regions can enhance the transferability of target attacks. The author provides theoretical analysis and experimental validation to support the rationale behind this hypothesis. Based on this hypothesis, the author designs a new generative attack method that improves the adversarial transferability of target attacks.

Strengths

* The idea of pushing ushing adversarial samples towards high-density regions in target attack is well-motivated * The theoretical analysis presented in this paper is insightful and solid.

Weaknesses

* The lack of persuasive experimental evidence is a major concern. 1. Insufficient ablation study: The paper introduces multiple modules in the proposed method, but the author fails to include an ablation study. Without such an analysis, it becomes challenging to ascertain whether the proposed method genuinely enhances adversarial transferability. For example, it would be valuable to investigate if the transferability improvement persists when pretrained embeddings are not utilized. 2. Limited impact of easy example-based adversarial attacks: The experiments presented in Table 1 demonstrate that the adversarial attacks based on easy examples have limited impact on transferability. Additionally, the experimental description in this section lacks sufficient detail. It would be helpful to clarify whether the baseline methods in Table 1 are based on cross-entropy. If they are, I recommend the author redefine the baselines, such as using target anchors generated from randomly selected samples of the target class. 3. Lack of validation experiments: Experiments are required to assess whether the adversarial examples generated by the proposed method exhibit higher density/smaller loss and loss gradients compared to those generated by baseline methods. It is essential for the author to include such validation experiments to provide empirical evidence supporting the claims regarding the improved density and reduced loss characteristics of the generated adversarial examples. * The analysis and proposed solution in the paper exhibit a gap. The conclusion drawn in the paper suggests that samples with lower loss and smaller gradients tend to have stronger transferability. A straightforward approach to address this would be to minimize not only the loss but also the gradient of the loss during the attack. It raises the question of why the author did not employ this direct and simple solution. Instead, the author chose to modify the loss to make the output of adversarial examples align with the target anchor. This approach introduces a gap between the previous analysis and the proposed solution. * Section 3 lacks clarity in its description. The role of pretrained embedding is not well-defined. I recommend the author to provide a detailed formula or explanation of how pretrained embedding is utilized in the generator.

Questions

* I suggest the authors conduct ablation studies to verify their claim. * Does making the output of adversarial examples match that of the target samples improve the density of adversarial examples?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors adequately addressed the limitations

Summary

The paper presents both theoretical and experimental evidence showing that incorporating perturbations towards the High-Sample-Density-Regions (HSDR) of the target class can significantly enhance the transferability of targeted attacks. To address the challenges of density estimation in high-dimensional data, a novel method for identifying samples within HSDR is proposed. Leveraging these insights, the authors introduce ESMA, which outperforms existing generative targeted attacks in terms of effectiveness and efficiency.

Strengths

The paper provides theoretical and experimental evidence to support the observation that deep learning models exhibit more consistent outputs in the High-Sample-Density-Regions (HSDR) of each class (Section 2.1). This finding establishes a foundation for the proposed approach. The paper demonstrates through experimental and theoretical analysis that easy samples with low loss in early-stopping models are commonly found in HSDR. Leveraging this insight, the proposed method directly adds perturbations targeting these samples of the target class, resulting in improved targeted adversarial transferability. This approach circumvents the need for computationally expensive density estimation of high-dimensional samples. Finally the paper introduces the Easy Sample Matching Attack (ESMA) which surpasses state-of-the-art generative attacks such as TTP in terms of targeted transfer success rate. Notably, ESMA only requires one model to perform attacks for all target classes significantly reducing storage space requirements and training time compared to TTP.

Weaknesses

There are some grammar mistakes (in lines 231-238, the sentences in the whole paragraph are all connected with commas, making it difficult to read) and the notion is a little bit confusing (e.g. in Algorithm 2, $a_k$ is in the feature space, but in the equation in line 251, $a_j$ is in the input space).

Questions

1 According to Algorithm 3, the target class is randomly sampled from a uniform distribution in the training stage. How does the generator generate a target adversarial example given a specific target class in the reference stage? 2 Is the model employed to select target anchors different from the source model in ESMA? This information is not clear.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

The writing and the usage of notation need to be enhanced.

Summary

This paper finds that optimizing perturbations towards easy examples located in HSDR can improve targeted adversarial transferability. The authors provide toy experiments and theoretical proof to illustrate these conclusions. They further utilize easy examples to guide the targeted attack in the GAN-like framework. This framework makes it feasible to perform the multi-target attack.

Strengths

1. Exploring targeted adversarial transferability in terms of High-Sample-Density-Regions is interesting. 2. The proposed framework support multi-target adversarial perturbation generation. 3. The motivation of this paper is clear.

Weaknesses

The experiments are insufficient. They miss previous works, hierarchical generative networks for multi-target attack [A], and iterative attacks [B,C,D, E]. Besides, the authors only conduct experiments on naturally trained models, and neglect to consider adversarially trained models and other defense methods. And the authors should attack more diverse network architectures, such as ViTs. [A] Yang X, Dong Y, Pang T, et al. Boosting transferability of targeted adversarial examples via hierarchical generative networks[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 725-742. [B] Zhao Z, Liu Z, Larson M. On success and simplicity: A second look at transferable targeted attacks[J]. Advances in Neural Information Processing Systems, 2021, 34: 6115-6128. [C] Qin Z, Fan Y, Liu Y, et al. Boosting the transferability of adversarial attacks with reverse adversarial perturbation[J]. Advances in Neural Information Processing Systems, 2022, 35: 29845-29858. [D] Wei Z, Chen J, Wu Z, et al. Enhancing the Self-Universality for Transferable Targeted Attacks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 12281-12290. [E] Byun J, Cho S, Kwon M J, et al. Improving the transferability of targeted adversarial examples through object-based diverse input[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 15244-15253.

Questions

Please conduct more experiments as discussed above.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

None.

Response to the rebuttal

Thanks for your rebuttal. For Concern1: [D] is the transferable targeted attack by jointly optimizing the global and local images, and it is not in the context of UAP scenarios. For Concern2: From Table 2 in the appendix, the performance of HGN and TTP is better than the proposed ESMA. The authors attribute this poor performance to the different distribution of adversarial examples. However, adversarially trained models also achieve high accuracy on benign inputs. Besides, the poor performance in attacking adversarially trained models exposes the vulnerability of the proposed method. For Concern3: Thanks for the results.

Comments on the authors' rebuttal

Thanks for your rebuttal. For concerns 2~4, thanks for the explanation and experiment and they well solve my concerns. For my concern 1, I know [3] does not provide any discussion or experiments in the context of target attack scenarios, but neither do SI-NIM, TIM and DIM. I wonder the peformance of [3] in targeted attack. Could you provide this experiment? BTW, For algorithm 3, what are the vaules of $N$ and $n$ set to? I can't find in the paper

Thanks for the response

The authors have addressed my questions, and I intend to keep my current rating.