Summary
This paper introduces the Backward Propagation Attack (BPA) to improve the transferability of adversarial examples generated based on the iterative gradient ascent procedure. The paper first empirically demonstrates that the non-linear layers such as ReLU and max-pooling truncate the loss gradient, and such a truncation procedure negatively affects the transferability of adversarial perturbations. Motivated by this observation, the paper proposes to modify the gradient computation for the ReLU and max-pooling layers. Particularly, consider a trained source model from which the adversarial perturbations are generated. BPA proposes to replace the derivative of the ReLU functions in the source model with the derivative of the SiLU function. Similarly, the derivative of the max-pooling function in the source model is replaced with a softmax function with a tunable temperature coefficient. Finally, results on randomly sampled 5000 Imagenet images demonstrate the improved transferability of adversarial perturbations generated based on PGD and MI-FGSM.
Strengths
Originality: Modifying the backward propagation procedure of the non-linear layers when computing the loss gradient during the process of generating adversarial examples is an interesting approach to improve the transferability of the perturbations.
Quality: The paper is well-written. The proposed BPA method is motivated using empirical observations. The evaluations include the transferability of perturbations under both targeted and untargeted settings.
Clarity: The motivation behind the proposed method as well as the structure of the paper is clear.
Significance: The adversarial robustness of deep learning models against transfer-based perturbations poses a practical security concern. This paper proposes a method that can be used in combination with existing gradient-based attack algorithms to improve the transferability of perturbations.
Weaknesses
The clarity of the paper can be improved. There are some terms that are key to understanding the message which the author wants to convey, but the lack of rigorous definitions makes it difficult to understand. (See Questions for a list of clarifications )
The evaluations in the paper are performed based on perturbations bounded by the L_inf norm with an $\epsilon$ of $\frac{8}{255}$. In Ln 242, it states that the choice of $\epsilon$ aligns with existing work (which one?), whereas most existing work considers $\frac{16}{255}$ (LBP, MI, VI/VMI, etc). Especially since one of the main contributions of the proposed method is the improvement of transferability over other sota methods (ie. "remarkable margin" ln284, ln296, ln 358), it is important to make sure the evaluations and the comparison with other methods are performed fairly.
Questions
Questions/clarifications:
In the abstract, why does the truncation make the gradient imprecise? The ReLU and the max-pooling layers are part of the network definition, so the derivative is exactly what it is supposed to be and I do not see the lack of precision. What does it mean by the relevance between gradient wrt input and the loss function? Since the term "relevance" appears later in the paper as well (eg. ln45, 58, 148, 315), having a more rigorous definition is warranted.
Figure 2 is a crucial element in motivating the proposed method. Is the success rate averaged over nine victim models? What is the variance among the results?
Ln 124 of Sec. 3.1: the paper first establishes that $z_i$ is the input to the function $f_i$ which returns $z_{i+1}$. When predicting the label, why is $z_l$ used as an input to the function $f_{l+1}$, shouldn't it be $z_{l+1}$?
Ln 138: Shouldnt the j-th element of $\frac{\partial z_{k+1}}{\partial z_k}$ be 1 if $f(z_{k})_j >0$?
Ln 139 "As a result, the gradient is effectively limited or weakened to some extent". ReLU was designed to address the vanishing gradient problem in training deep models. By having a step function as its derivative, it allows some gradient information to pass freely while simultaneously not updating other weights. Therefore, this is exactly how the backward propagation was designed to be with ReLU-activated networks.
Ln 414: Similarly, this is exactly how bp is supposed to behave with max-pooling.
Ln 149: What does it mean by "decaying the relevance"?
Ln 185-186: Is this a prior result? or a claim made in this paper supported by evidence?
Suggestions: Could we replace the y-axis ("Attack success rate") with the adversarial loss value on the source model? That is to plot $\ell(x+\Delta x)$, where $\Delta x$ is generated using fgsm/i-fgsm/mi-fsgm on the source model modified using the various method. If we observe a higher loss value after the modification, it indeed shows that the derivative of those nonlinear layers prevents the perturbations from reaching maxima. On the other hand, if we observe a lower loss value after the modification, this means the truncation effect of the derivative of the non-linear functions leads to perturbations overfit to the source model, despite having higher losses.
Other minor suggestions:
Figure 1 should not come after Figure 2.
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Limitations
There is no discussion of the limitation. For instance, the proposed paper is only suitable for attack algorithms that are based on gradient computations.