Transferability Bound Theory: Exploring Relationship between Adversarial Transferability and Flatness

Response

Thank you for your feedback. We have explicitly made the assumptions ($p(x+\delta) \leq p(x)$ and $F'(x+\delta) \geq F(x+\delta)$) in the revised manuscript to avoid any potential confusion. To clarify, $F'(x+\delta)$ represents the probability that the target model correctly classifies $x+\delta$, i.e., $0 \leq F'(x+\delta) \leq 1$. Regarding $F'(x+\delta) \geq F(x+\delta)$ and the confusion surrounding Equation 12, we have provided a detailed explanation in our response to Reviewer x5ig. Please refer to that for further details. Concerning $p(x+\delta) \leq p(x)$, there is substantial supporting evidence from literature [1,2,3] and practical considerations. Adversarial examples are artificially crafted through specific algorithms and optimization processes, tailored to particular models and tasks. In practice, the distributions we encounter are typically those generated by the natural processes that generate the data, and these do not favor adversarial examples over natural ones. As such, they are not naturally occurring, and their probability of appearance in the real world is significantly lower compared to natural samples. Formally, let us consider the following derivation: $$ p(x+\delta) = \sum p(x+\delta, y_i) = \sum p(y_i) p(x+\delta|y_i) = \sum p(y_i) p(x|y_i) + \sum p(y_i) \nabla p(x|y_i)^T \delta = p(x) + \sum p(y_i) \nabla p(x|y_i)^T \delta. $$ Since the ground-truth label $y_g$ for a specific natural sample $x$ is fixed, we can simplify the above equation to: $$ p(x+\delta) = p(x) + p(y_g) \nabla p(x|y_g)^T \delta. $$ Intuitively, $\delta$ should satisfy $p(x|y_g)^T \delta \leq 0$, which means that $\delta$ does not increase the probability of the ground-truth label $y_g$ after being applied to $x$. This is because $\delta$ is generated by a proxy model, which is trained on the data distribution $p(x,y)$. Thus, $p(x|y_g)^T \delta > 0$ is counterintuitive and unlikely, unless the proxy model has learned a distribution that is negatively associated with $p(x,y)$. In practice, DNNs, especially those used in real-world applications, perform well on $x$. For poorly performing DNNs, they may already have low accuracy on $x$, and thus generating adversarial examples for them is trivial. Therefore, the assumption that $p(x+\delta) \leq p(x)$ is reasonable. We hope these clarifications help to strengthen the rationale behind our assumptions and provide a clearer understanding of the context. We also highlight that this manuscript serves as the first theoretical study on adversarial example transferability, and we believe it can offer the community deeper insights into understanding adversarial example transferability. We look forward to your reply and hope that this addresses your concerns. --- [1] On the (Statistical) Detection of Adversarial Examples [2] Out-of-Distribution Data: An Acquaintance of Adversarial Examples -- A Survey [3] Interpreting Adversarial Examples in Deep Learning: A Review

Response (1/2)

We appreciate your feedback. We would like to provide further clarification as we sense some misunderstandings here. We also look forward to your reply and hope that this addresses your concerns. --- **Question:** Our bound. **Response:** Notice that the expected square loss of the generated adversarial examples on the target model (DenseNet121) is $10.50^2$. In statistical terms, while individual sample's squared loss may vary around this expected value, we expect that the majority of samples will exhibit comparable losses. To be more specific, we empirically evaluate the variance of the squared losses to be 54.62 (out of 1000 adversarial examples generated by our attack). According to Chebyshev's inequality, at least 96\% of the samples should lie within 5 standard deviations of the expected value. Within this range, the squared losses of our generated adversarial examples should be at least $10.50^2 - 5 \times \sqrt{54.62} = 73.30 \approx 8.56^2$. This implies that our generated adversarial samples incur a loss greater than 8.56 on DenseNet121 with at least 95\% probability. A loss of 8.56 indicates that the probability of correct classification by DenseNet121 for these samples is approximately 0.0002. In other words, out of every hundred samples, at least 95 can effectively mislead the target model. Doesn't this high probability of successfully attacking the target model sufficiently demonstrate the effectiveness and practicality of our bound? --- **Question:** The effectiveness of TPA. **Response:** As stated in our introduction, inspiring works such as [1, 11, 41] prompt us to investigate the theoretical relationship between flatness and the transferability of adversarial examples. Our theory suggests that penalizing the first and second-order gradients of generated adversarial examples can effectively enhance their transferability. Notably, prior works [1, 11, 41] did not include penalties on second-order gradient. Our proposed method is simple yet quite effective, which can generate more transferable adversarial examples by penalizing both one-second and second-order gradients via additional noise. We acknowledge the contributions made by these existing works in our introduction and clarify the distinctions between our method and theirs.

Response(2/2)

**Question:** The assumption about $F'(x+\delta) \geq F(x+\delta)$. **Response:** Regarding the second concern, there are some typos in the appendix of the original paper. Let us consider the following derivation: $$ \sum \int p(x) \Vert \delta \Vert _2^2 \nabla F'(x+\delta)[i] \cdot \nabla \log F(x+\delta)[i] dx $$ $$ = \sum p(x) \Vert \delta \Vert_2^2 F'(x+\delta) \cdot \nabla \log F(x+\delta)[i] |_a^b - \sum \int p(x) \Vert \delta \Vert_2^2 F'(x+\delta) \nabla^2 \log F(x+\delta)[i,i] dx $$ $$ = - \sum \int p(x)\Vert \delta \Vert_2^2 F'(x+\delta) \nabla^2 \log F(x+\delta)[i,i] dx \geq - \sum \int p(x) \Vert \delta \Vert_2^2 F(x+\delta) \nabla^2 \log F(x+\delta)[i,i] dx,$$ where $a=+\infty, b=-\infty$. The second equality utilizes integration by parts, the third equality uses $p(\infty)=0$ and the fourth equality leverages $F'(x+\delta) \geq F(x+\delta)$ and the fact that the second derivative of $\log$ is negative. This expression indeed derives $\int p(x) \Vert \delta \Vert_2^2 \nabla \log F'(x+\delta)^\top \nabla \log F(x+\delta) dx \geq - \sum_{i} \int p(x) \Vert \delta \Vert_2^2 F(x+\delta) \nabla^2 \log F(x+\delta)[i,i] dx.$ We rely on $F'(x+\delta) \geq F(x+\delta)$, a generally valid assumption. $\delta$ typically maximizes the loss function on the proxy model, implying $F'(x+\delta)$ optimally suits the proxy, i.e., $F'(x+\delta) \geq F(x+\delta)$. Moreover, we conduct experiments using BIM and our method to generate adversarial examples for 1000 natural samples on ResNet50. For DenseNet121, EfficientNet, VGG19, ConvNet, and Vision Transformer (ViT), all correctly predict these generated adversarial examples with higher confidence compared to the proxy model, meaning they assign a higher prediction probability to the ground-truth class. Specifically, in the presence of adversarial examples generated by our attack, DenseNet121, EfficientNet, VGG19, ConvNet, and ViT have correct prediction probabilities that are 61 times, 197 times, 406 times, 1058 times, and 3674 times higher, respectively, than those of the proxy model ResNet50 (note that since the proxy model's correct prediction probability for adversarial examples is typically very tiny, such as around $10^{-7}$, the target models are still easily misled by these adversarial examples). Thus, assuming $F'(x+\delta) \geq F(x+\delta)$ in transfer attack scenarios is justified. Furthermore, even if we consider the existence of peculiar samples satisfying $F'(x+\delta) > F(x+\delta)$, there is no need to investigate these further because $F'(x+\delta)>F(x+\delta)$ already suggests that $x+\delta$ can trick the target model (since adversarial examples generated on the proxy model often mislead the proxy itself). In other words, transfer-based attacks are meaningful only if $F'(x+\delta) \geq F(x+\delta)$. For theoretical rigor, we can adjust our analysis as follows: $$ - \sum \int p(x)\Vert \delta \Vert_2^2 F'(x+\delta) \nabla^2 \log F(x+\delta)[i,i] dx = - \sum \int_{U_1} p(x)\Vert \delta \Vert_2^2 F'(x+\delta) \nabla^2 \log F(x+\delta)[i,i] dx - \sum \int_{U_2} p(x)\Vert \delta \Vert_2^2 F'(x+\delta) \nabla^2 \log F(x+\delta)[i,i] dx$$ $$ \geq - \sum \int_{U_1} p(x)\Vert \delta \Vert_2^2 F'(x+\delta) \nabla^2 \log F(x+\delta)[i,i] dx \geq - \sum \int_{U_1} p(x)\Vert \delta \Vert_2^2 F(x+\delta) \nabla^2 \log F(x+\delta)[i,i] dx. $$ Here, we split the entire integration domain into two non-overlapping parts: $U_1$ where $F'(x+\delta) \geq F(x+\delta)$ and $U_2$ where $F'(x+\delta) < F(x+\delta)$. Due to $- \sum \int_{U_2} p(x)\Vert \delta \Vert_2^2 F'(x+\delta) \nabla^2 \log F(x+\delta)[i,i] dx \geq 0$, we can disregard this term. This adjustment does not compromise the integrity of our theory; it merely restricts the integration interval to $U_1$. We hope this can address your concerns.

After the rebuttals

Thank the authors for your rebuttals carefully. My concerns about unclear details and experiments are well addressed. Thus, I would like to raise my final score. Besides, the assumptions must be explicitly presented and explained in the final version.

Looking forward to your feedback

Dear Reviewer x5ig, Sorry to bother you again. With the discussion phase nearing the end, we would like to know whether the responses have addressed your concerns. Should this be the case, we are encouraged that you raise the final rating to reflect this. If there are any remaining concerns, please let us know. We are more than willing to engage in further discussion and address any remaining concerns to the best of our abilities. We are looking forward to your reply. Thank you for your efforts in this paper. Best regards, Authors

Looking forward to your feedback

Dear Reviewer QVii, Sorry to bother you again. With the discussion phase nearing the end, we would like to know whether the responses have addressed your concerns. Should this be the case, we are encouraged that you raise the final rating to reflect this. If there are any remaining concerns, please let us know. We are more than willing to engage in further discussion and address any remaining concerns to the best of our abilities. We are looking forward to your reply. Thank you for your efforts in this paper. Best regards, Authors