Random Logit Scaling: Defending Deep Neural Networks Against Black-Box Score-Based Adversarial Example Attacks
Machine learning models are increasingly adapted in various domains. However, adversarial examples pose a significant threat to the reliable deployment of these models. In recent years, some powerful adversarial example attacks have been proposed for the fast and query-efficient generation of adversarial examples, even in black-box scenarios, highlighting the need for scalable, low-cost, and powerful defenses. In this work, we present two contributions to the domain of black-box adversarial example attacks and defenses. First, we propose Random Logit Scaling (RLS), a randomization-based defense against black-box score-based adversarial example attacks. RLS is a plug-and-play, post-processing defense that can be implemented on top of any existing ML model with minimal effort. The idea behind RLS is to confuse an attacker by outputting falsified scores resulting from randomly scaled logits while maintaining the model accuracy. We show that RLS significantly reduces the success rate of state-of-the-art black-box score-based attacks while preserving the accuracy and minimizing confidence score distortion compared to state-of-the-art randomization-based defenses. Second, we introduce a novel adaptive attack against AAA, a SOTA non-randomized black-box defense against black-box score-based attacks that also modifies output logits to confuse attackers, demonstrating its vulnerability against adaptive attacks.
Paper
References (45)
Scroll for more · 33 remaining
Similar papers
Peer review
Summary
Based on the idea that a defender could blind an attacker by not producing true logit values, resulting in falsified confidence scores, this paper proposes Random Logit Scaling (RLS) as a novel defense mechanism. While random logit scaling could degrade model performance, RLS mitigates this by scaling output logits with a carefully chosen positive factor $m$, effectively reducing attack success rates, and preserving clean accuracy. Experimental results demonstrate the effectiveness of this approach.
Strengths
1. Motivation and observation are both clear. 2. This work is a post-processing method. doesn't need any overhead during training. This is important for larger models and datasets. 3. The experiments span a range of attacks and datasets, providing strong empirical support for the robustness and effectiveness of RLS.
Weaknesses
1. In comparison with other baselines, it’s straightforward in your framework to explain why they lead to performance degradation. However, it’s less clear why your method achieves a lower attack success rate than others. Could you clarify the limitations of the other methods? 2. The choice of using a uniform distribution for logit scaling, though effective, appears somewhat arbitrary. I think more discussion on potential alternative distributions would strengthen the approach. 3. In the experiments, a few fixed settings are explored for the range of $m$. I am curious about how varying the range of $m$ impacts model robustness and accuracy. An empirical ablation study on optimal parameter selection would add valuable insights. 4. I'm just curious if this method can be applied in the scenario of a white box attack.
Questions
See above.
Rating
6
Confidence
3
Soundness
2
Presentation
3
Contribution
3
Authors' Response to Reviewer GqXN
We thank you for your review and feedback and hope to address your questions and concerns effectively in the following. **Q1: In comparison with other baselines, it’s straightforward in your framework to explain why they lead to performance degradation. However, it’s less clear why your method achieves a lower attack success rate than others. Could you clarify the limitations of the other methods?** **Response:** RLS unpredictably alters the values of the objective loss function targeted by an attacker by directly and randomly modifying the logit values. This results in a highly unsmooth loss landscape, making it difficult for attackers to identify reliable optimization directions (as illustrated in Fig. 2.a of our paper). In contrast, iRND introduces small noise to the input to disrupt the noise used by attackers to estimate the gradient of the loss function. Similarly, RFD and oRND randomize model outputs by adding Gaussian noise to intermediate features or model outputs. However, the randomization introduced by these defenses causes only minor changes to the loss landscape compared to RLS. Consequently, attackers can still successfully craft adversarial examples, as some attack iterations may still identify valid optimization directions. **Q2: The choice of using a uniform distribution for logit scaling, though effective, appears somewhat arbitrary. I think more discussion on potential alternative distributions would strengthen the approach.** **Response:** In our original submission, we addressed the choice of distribution in Section 5.2. We initially experimented with Gaussian and uniform distributions, with the results presented in Table 1 of our work. One limitation of using a Gaussian distribution over a uniform distribution with RLS is the additional step required to ensure that sampled values are non-negative. Moreover, the differences in defense performance between the two distributions were minimal, with Gaussian showing a slight advantage (approximately 4% added robustness in certain configurations). Nevertheless, we selected the uniform distribution for the remainder of our experiments because of its simplicity. **Q3: In the experiments, a few fixed settings are explored for the range of $m$. I am curious about how varying the range of $m$ impacts model robustness and accuracy. An empirical ablation study on optimal parameter selection would add valuable insights.** **Response:** We evaluated five different configurations for the range of **m**, as reported in Section 5.2 and Table 1 of our original submission. Our findings indicate that both decreasing the lower bound and increasing the upper bound can significantly reduce the attack success rate. For example, the range $(0.1,100)$ demonstrated the highest robustness among the configurations tested. However, we selected a milder configuration with a range of $(0.5,10)$ and a more robust configuration of $(0.5,1000)$ for our experiments since we observed that RLS still consistently achieves higher robustness compared to other defenses across various models and attacks with both of these configurations, showing minimal sensitivity to parameter choices. This contrasts with defenses like RFD, which require fine-tuning specific parameters (e.g., the $\sigma$ value) for each model being defended. **Q4: I'm just curious if this method can be applied in the scenario of a white box attack.** **Response:** In a white-box scenario, the attacker has full access to the target model and can potentially bypass or remove any integrated defenses, such as RFD, iRND, oRND, AAA, or our proposed method, RLS. However, for the purpose of this evaluation, we assume that the attacker does not alter or remove the defense mechanisms and performs a white-box attack directly on the defended model. To conduct these attacks, we utilize the PGD method [1], with the results presented in the table below: *Accuracy under the white-box PGD attack on ResNet-18 model*. | Defense | Accuracy | | --------------- | -------- | | Undefended | 2.11% | | RLS (0.5, 10) | 7.40% | | RLS (0.5, 100) | 14.00% | | RLS (0.5, 1000) | 15.50% | [1] Aleksander Madry, et al. Towards Deep Learning Models Resistant to Adversarial Attacks. ICLR, 2018.
We hope to have addressed your main concerns, including *the choice of distribution* for RLS and the parameter *values of $m$*. In our uploaded revised paper, we have highlighted and clarified the study on the choice of the distribution and its parameters in **Section 5.2.2 (highlighted in red)** to further address your concerns. The results are now presented in a dedicated section **Appendix D**. Additionally, we hope we have addressed your inquiry regarding the application of RLS in *white-box settings* and provided a clearer explanation of RLS's *advantages compared to other methods*. We thank you for your feedback.
Summary
This paper explores a new randomization-based method Random Logit Scale (RLS) to defend against black-box score-based adversarial attacks. RLS is a post processing defense which confuses an attacker by randomly scaling the output logit. While maintaining the model clean accuracy and confident score distortion, RLS can significant reduce the attack success rate of SOTA black-box score-based attacks.
Strengths
- The paper's motivation is clear and convincing. The idea behind why randomly scaling the model logits helps fortify against black-box score-based attacks sounds intuitively reasonable. - Most figures are easy to understand and the writing is clear and easy to follow. - The experiments are done against popular black-box attacks and the empirical improvements compared to baselines seem significant and no performance drop is desirable.
Weaknesses
My main concerns are: 1. Lack of theoretical analysis 2. Some results in the experiments are not convincing 3. Some parts of the paper are not well written. Please see the section on questions for a list of my concerns.
Questions
## Main questions: 1. The method is very simple and straightforward. Randomly scaling the output logits does not hurt the accuracy because it maintains the rank of predicted labels if $m$ is larger than 0 and is able to falsify the output scores and confuse an adversary. This approach is also applicable for output softmax, so how does it work if the output softmax is randomly scaled instead of output logits? 2. Although the authors provided an analysis of the impact of RLS on the confidence scores, a theoretical analysis of the efficiency and robustness of RLS against score-base attacks, as in [1, 2], is missing. I encourage the authors to provide a similar theoretical analysis. 3. $l_2, l_\infty$ and $l_0$ attacks are different, and a defence designed for score-based attacks should be robust against different $l_p$ attacks. I would suggest the authors should conduct more comprehensive experiments across different $l_p$ score-based attacks [3, 4]. 4. The larger the perturbation is, the higher the attack success rate of an attack is, and there is no guarantee RLS is better than other defence mechanisms under larger perturbation attacks. Therefore, a strong defense should work well across different perturbation budgets and I encourage the authors demonstrates the robustness of RLS vs others across different perturbation budgets. 5. The average number of queries of different attacks against different defense methods is very low if compared with the query budget used in this study (10,000). This means that these attacks successfully yield adversarial examples for most of the selected samples. The average number of queries should have a correlation with the ASR. For instance, lower ASR attacks should have an approximate or higher average number of queries than higher ASR attacks. However, results in Tables 3 and 4 show that the average number of queries of Bandit and Square attacks against RLS is significantly low and even lower than the non-defensive method, while ASR is extremely low if compared with the non-defensive method (~100% ASR). 6. The experiment setup for RFD method reported in Table 3 and 4 is not consistent. The values of σ are varied and the accuracy drop is not consistent around 1% or 2% as reported in [2]. For a fair comparison, the authors should evaluate RFD with the same settings as done in [2] when the clean accuracy drop is around 1% or 2%. 7. As some results in the paper are not convincing and evaluation settings are not comprehensive mentioned, I suggest the authors release your implementation for replication and verification purposes? 8. In this study, EOT is employed as an adaptive attack and it can be incorporated with gradient-estimation or random search attack method. Although the authors use k queries for each sample at each attack iteration and take an average over the outputs to cancel out the randomness but it is unclear what method is used to determine attack direction e.g. gradient-based or random search approach. What is the perturbation budget used to attack against the adaptive attack? Again, the query average is very low if compared to the query budget but the ASR is not high. Can the author report the median query number instead and provide a ridiculous explanation? 9. The authors discussed AAA defense which is designed for score-based attacks and this defense does not induce an accuracy drop as RLS. For completeness, I encourage the author to evaluate and compare this defense with RLS. ## Minor: 1. There are some contributions that are not actually contributions, e.g., 2 and 3. The authors should reorganize and summarize contributions 1 and 4 more thoroughly. 2. Is minimizing distortion to the confidence score (in the introduction and section 5.3) really worth achieving? If so, the authors should cite papers or provide existing applications that rely on the accuracy of the gap between the model’s top and runner-up classes’ confidence scores to support your argument. 3. The experimental setup is not well explained and lacks some crucial information. For instance, what is the $l_p$ norm of the attacks? What is the perturbation budget used in this study? How many samples are selected for evaluation on CIFAR-10? How did the author select these samples? The author should clearly mention how samples are selected for evaluation on ImageNet in the experimental setup, e.g. 1000 samples are randomly selected, as mentioned in the caption of Table 4 in the experimental setup. Similarly, 2000 samples from CIFAR-10 are in the caption of Table 5. [1] Zeyu Qin, Yanbo Fan, Hongyuan Zha, and Baoyuan Wu. Random noise defense against query-based black-box attacks. NeurIPS, 2021. [2] Nguyen Hung-Quang, Yingjie Lao, Tung Pham, Kok-Seng Wong, and Khoa D. Doan. Understanding the robustness of randomized feature defense against query-based adversarial attacks. ICLR, 2024 [3] Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square Attack: a query-efficient black-box adversarial attack via random search. ECCV, 2020 [4] Quoc Viet Vo, Ehsan Abbasnejad, and Damith C. Ranasinghe. Brusleattack: Query-efficient score-based sparse adversarial attack. ICLR, 2024.
Rating
5
Confidence
4
Soundness
2
Presentation
3
Contribution
2
Authors' Response to Reviewer wBXU (1/6)
We thank you for your valuable feedback and insights and hope to address your concerns and questions effectively in the following. **Q1: The method is very simple and straightforward. Randomly scaling the output logits does not hurt the accuracy because it maintains the rank of predicted labels if $m$ is larger than 0 and is able to falsify the output scores and confuse an adversary. This approach is also applicable for output softmax, so how does it work if the output softmax is randomly scaled instead of output logits?** Scaling or perturbing output softmax as a defense is more complex than scaling logits. Adjusting the softmax output requires additional steps to make sure that two criteria is met: 1. The softmax output values sum to 1 2. The ranking of the classes remain unchanged Although each criterion can be straightforward to achieve independently, fulfilling both of these objectives simultaneously requires careful consideration and added complexity in the choice of perturbation applied to the softmax output. In contrast, scaling logits by a positive value, automatically satisfies both of criteria. Additionally, RLS can be interpreted as equivalent to adding a vector $v$ to output softmax that satisfies the two criteria outlined earlier, and therefore we expect it to provide the same defense robustness. Assuming we have $n$ classes and our original output logit is the vector $z$ it can be shown that scaling logits by $m$ is equivalent to addinga vector $v$ to the output softmax $$ \begin{split} & \text{softmax}(z\times m) = \text{softmax}(z) + v \\ & v = \text{softmax}(z\times m) - \text{softmax}(z) \\ \end{split} $$ we can write $\text{softmax}(z_i \times m)$ as, $$ \begin{split} \text{softmax}(z_i\times m) &= \frac{e^{m\times z_i}}{\sum_{j=1}^{n}e^{m\times z_j}} \\ &= e^{(m-1)\times z_i}\times\frac{e^{z_i}}{\sum_{j=1}^{n}e^{z_j}}\times\frac{{\sum_{j=1}^{n}e^{z_j}}}{\sum_{j=1}^{n}e^{m\times z_j}} \\ &= \text{softmax}(z_i) \times e^{(m-1)\times z_i}\times \frac{{\sum_{j=1}^{n}e^{z_j}}}{\sum_{j=1}^{n}e^{m\times z_j}} \end{split} $$ therefore, $v_i$ is: $$ v_i = \text{softmax}(z_i) \times [e^{(m-1)\times z_i} \times \frac{\sum_{j=1}^{n} e^{z_j}}{\sum_{j=1}^{n}e^{m\times z_j}}-1] $$ where $-1 \leq v_i \leq 1$. **Q2: Although the authors provided an analysis of the impact of RLS on the confidence scores, a theoretical analysis of the efficiency and robustness of RLS against score-base attacks, as in [1, 2], is missing. I encourage the authors to provide a similar theoretical analysis.** Following RFD [2] we can formulate our defense as simply adding a vector $v$ to the output of the softmax layer, where for each $v_i \in v$ we have $-1 \leq v_i \leq 1$ (as detailed in the response to your first question). However, our attempts at conducting a qualitative analysis analogous to [1] or [2] were unsuccessful due to the fact that the randomness of RLS does not exhibit the same smooth behavior as the randomized parameters in RFD or iRND. Even using a Gaussian distribution, RLS requires additional modifications to ensure that only positive values are sampled. Consequently, any qualitative analysis of RLS necessitated significant approximation or oversimplification. To address this challenge, we focused on demonstrating the defense's effectiveness through a series of comprehensive experiments, employing a wide range of attacks and models.
Authors' Response to Reviewer wBXU (2/6)
**Q3: $\ell_2, \ell_\infty, \ell_0$ attacks are different, and a defence designed for score-based attacks should be robust against different $\ell_p$ attacks. I would suggest the authors should conduct more comprehensive experiments across different $\ell_p$ score-based attacks.** Thank you for your suggestion. In our initial experiments we focused on $\ell_\infty$ attacks due to their demonstrated stronger attack performance. We will revise Section 5 of our paper to make sure this is stated clearly. We have now additional results in the $\ell_2$ norm space, and our results indicate that RLS consistently outperforms other defenses similar to the findings in the $\ell_\infty$ norm space. We conduct experiments on ResNet-18 and VGG, using Bandit, Square, and ZoSignSGD attacks. The results are presented in the tables below. We use the same set of 1,000 randomly selected test samples from CIFAR-10 to conduct these experiments. We will revise our paper to include these results in the Appendix section. *$l_2$ results for ResNet-18 over the randomly selected set of 1,000 test samples from CIFAR-10.* | Defense Method | Bandit | Square | ZoSignSGD | | --------------------- | -------------------- | --------------------------- | -------------------- | | No Defense | 98.76% / 961.32 | 97.83% / 830.28 | 56.02% / 694.10 | | iRND ($\nu = 0.01$) | 14.52% / 1502.34 | 32.39% / 2060.240 | 50.69% / 935.96 | | iRND ($\nu = 0.02$) | 25.66% / 1668.24 | 34.98% / 1460.73 | 41.56% / 1116.96 | | oRND ($c=1$) | 17.60% / 2136.53 | 26.29% / 1395.87 | 45.24% / 925.85 | | RFD ($\sigma=2.5$) | 24.25% / 2125.35 | 30.81% / 1855.04 | 44.44% / 953.40 | | RLS (0.5, 10) | **5.92% / 1709.42** | 25.32% / 5189.30 | 36.01% / 867.03 | | RLS (0.5, 1000) | 6.02% / 1964.12 | **9.65% / 658.78** | **34.86% / 948.09** | *$l_2$ results for VGG-16 over the randomly selected set of 1,000 test samples from CIFAR-10.* | Defense Method | Bandit | Square | ZoSignSGD | | --------------------- | ------------------------- | ------------------------- | -------------------- | | No Defense | 98.07% / 1463.61 | 96.23% / 2054.00 | 30.29% / 681.56 | | iRND ($\nu = 0.01$) | 8.08% / 1657.52 | 29.07% / 3604.49 | 28.85% / 736.11 | | iRND ($\nu = 0.02$) | 14.57% / 1355.37 | 20.44% / 1922.56 | 24.79% / 1155.64 | | oRND ($c=1$) | 11.87% / 2146.18 | 18.07% / 1922.56 | 26.45% / 1119.60 | | RFD ($\sigma=0.35$) | 28.02% / 1464.05 | 30.14% / 1488.91 | 29.36% / 1318.08 | | RLS (0.5, 10) | 3.78% / 1100.05 | 16.60% / 4926.59 | 20.80% / 906.11 | | RLS (0.5, 1000) | **3.56% / 1347.45** | **7.55% / 1627.91** | **19.40% / 816.35** | We can see that RLS still outperforms other defense methods in the $\ell_2$ space for Square, Bandit, and ZoSignSGD.
Authors' Response to Reviewer wBXU (3/6)
**Q4: The larger the perturbation is, the higher the attack success rate of an attack is, and there is no guarantee RLS is better than other defence mechanisms under larger perturbation attacks. Therefore, a strong defense should work well across different perturbation budgets and I encourage the authors demonstrates the robustness of RLS vs others across different perturbation budgets.** We apologize for the oversight in providing a detailed explanation of the experimental setup. We will revise Section 5.1 and Appendix A of our paper to include a comprehensive overview of the experimental setup. The results presented in our original submission were based on a perturbation size of $0.05$ ($12.75/255$). We have now conducted additional experiments with a larger perturbation size of $0.1$ ($25.5/255$), and the results confirm that RLS continues to outperform other defenses. These experiments were conducted using the same set of 1,000 randomly selected test samples from the CIFAR-10 dataset. We use VGG-16 and ResNet-18 architectures along with Square, SignHunter, and Bandit attacks: *Attack success rate and average number of queries for ResNet-18 on the random 1,000 test samples from CIFAR-10 using 0.1 perturbation size.* | Defense Method | Square | SignHunter | Bandit | | --------------------- | ------------------------ | ------------------------- | ------------------------ | | No Defense | 100% / 19.52 | 100% / 118.04 | 100% / 61.65 | | iRND ($\nu = 0.01$) | 99.69% / 52.35 | 97.69% / 274.79 | 90.54% / 319.45 | | iRND ($\nu = 0.02)$ | 97.59% / 68.24 | 89.37% / 446.57 | 90.90% / 194.98 | | oRND ($c=1$) | 96.44% / 69.75 | 68.91% / 770.67 | 93.72% / 132.82 | | RFD ($\sigma=2.5$) | 96.68% / 127.34 | 67.96% / 786.64 | 96.05% / 196.47 | | RLS (0.5, 10) | 93.78% / 357.41 | **40.77% / 787.85** | **55.50% / 37.90** | | RLS (0.5, 1000) | **85.48% / 37.84** | **36.93% / 768.46** | **0.52% / 3.0** | *Attack success rate and average number of queries for VGG-16 on the random 1,000 test samples from CIFAR-10 using 0.1 perturbation size.* | Defense Method | Square | SignHunter | Bandit | | --------------------- | ------------------------- | ------------------------- | ------------------------ | | No Defense | 100.00% / 37.55 | 100% / 92.47 | 100% / 85.82 | | iRND ($\nu = 0.01$) | 99.79% / 48.70 | 99.90% / 117.21 | 62.11% / 678.10 | | iRND ($\nu = 0.02)$ | 98.48% / 93.05 | 98.59% / 181.11 | 61.42% / 392.69 | | oRND ($c=1$) | 88.14% / 122.22 | 81.07% / 208.68 | 60.72% / 243.98 | | RFD ($\sigma=0.35$) | 89.74% / 323.78 | 84.94% / 483.64 | 74.45% / 318.79 | | RLS (0.5, 10) | **72.74% / 581.40** | **48.50% / 476.71** | **31.15% / 25.55** | | RLS (0.5, 1000) | **66.92% / 28.54** | **52.27% / 321.19** | **1.73% / 8.81** |
Authors' Response to Reviewer wBXU (4/6)
**Q5: The average number of queries of different attacks against different defense methods is very low if compared with the query budget used in this study (10,000). This means that these attacks successfully yield adversarial examples for most of the selected samples. The average number of queries should have a correlation with the ASR. For instance, lower ASR attacks should have an approximate or higher average number of queries than higher ASR attacks. However, results in Tables 3 and 4 show that the average number of queries of Bandit and Square attacks against RLS is significantly low and even lower than the non-defensive method, while ASR is extremely low if compared with the non-defensive method (~100% ASR).** In our experiments, we observed cases where RLS significantly reduces the Attack Success Rate (ASR) without substantially increasing the average number of queries, particularly in the case of the Square and Bandit attacks. This observation can be explained as follows: Lower ASR can be caused by "hard" samples: For both Bandit and Square attacks, RLS reduces the success rate by at least ~60%. This indicates that these attacks are unable to craft adversarial examples for a significant proportion of the selected samples. We observed that most of these unsuccessful samples remain unaffected throughout the 10,000-query budget. However, for a smaller proportion of samples, the initial steps of the attacks are still able to produce successful adversarial examples (e.g., within the first 100 queries). Specifically, the initial step of the Square attack, which uses vertical stripe perturbations achieves an ASR of ~30% against an undefended ResNet-18 model (on the 1,000 random samples of CIFAR-10). However, in the presence of RLS, the ASR during this step is ~60% meaning that the majority of the samples require only a couple of queries for a successful attack. In contrast, for weaker defenses and the undefended model, some of these "hard-to-evade" samples eventually become susceptible to attack later within the query budget (e.g., after 2,000 queries) thereby increasing the average number of queries. We provide additional numerical data to illustrate this point. We run Square attack on ResNet-18 using the same randomly selected 1,000 test samples from CIFAR-10 under two conditions: the undefended ResNet-18 and ResNet-18 with RLS for $a=0.5$ and $b=1000$. Compared with the undefended model, RLS has ~30 fewer average queries while achieving a ~38% higher robustness against the attack. The distribution of the number of queries for successfully crafted adversarial example is as follows: *Distribution of the number of queries for successfully crafted adversarial example for Square against undefended ResNet-18 and ResNet-18 with RLS (0.5, 1000).* | Defense | 2 Queries (initial vertical strip perturbation) | $\leq 50$ Queries | $\leq 100$ Queries | ASR | | --------------- | :---------------------------------------------: | :-----------------: | :------------------: | ---- | | Undefended | 25.10% | 67.22% | 82.57% | 100% | | RLS (0.5, 1000) | 62.53% | 90.70% | 94.32% | ~43% | We performed a similar analysis for the Bandit attack on an undefended VGG-16 model and a VGG-16 model with RLS for $a=0.5$ and $b = 10$ and report the distribution of the query counts as follows: *Distribution of the number of queries for successfully crafted adversarial example for Bandit against undefended VGG-16 and VG-16 with RLS (0.5, 10).* | Defense | $\leq 10$ Queries | $\leq 50$ Queries | $\leq 100$ Queries | ASR | | ------------- | :-----------------: | :-----------------: | :------------------: | ---- | | Undefended | 10.67% | 32.97% | 51.40% | 100% | | RLS (0.5, 10) | 45.45% | 89.09% | 97.27% | ~11% | Other randomization-based defenses, make it harder, but not impossible for crafting adversarial examples meaning that at some iteration later in the query budget, the attacker may eventually find multiple correct steps to create an adversarial example, hence increasing the average number of queries. However, RLS makes it impossible for the majority of the samples to be made evasive within the full 10,000-query budget.
Authors' Response to Reviewer wBXU (5/6)
**Q6: The experiment setup for RFD method reported in Table 3 and 4 is not consistent. The values of σ are varied and the accuracy drop is not consistent around 1% or 2% as reported in [2]. For a fair comparison, the authors should evaluate RFD with the same settings as done in [2] when the clean accuracy drop is around 1% or 2%.** For RFD, the value of $\sigma$ needs to be adjusted for each model, hence the values we have presented in Table 3 and 4 are varied and are model specific. Additionally, the original RFD paper does not explicitly provide $\sigma$ values but instead report the average accuracy drop. The authors report Attack Success Rates (ASR) corresponding to average accuracy drops of approximately 1%, 2%, 3%, or 4%. Therefore, we had to determine the optimal $\sigma$ values for each of our models. We performed a parameter search for each model and selected the best-performing σ which is reflected in the results reported in Tables 3 and 4. The table below shows some of the $\sigma$ values we evaluated. We would also like to highlight that **this represents another weakness of RFD compared to our proposed method.** RFD requires manual tuning of its parameters for each model, whereas RLS demonstrates consistent performance across various attacks and models with minimal sensitivity to its parameter values. RLS consistently outperforms other defense methods across all tested victim models, without requiring parameter adjustments and with negligible additional computational overhead. *Performance of various $\sigma$ values for ResNet-18 against Square attack. We chose $\sigma=2.5$ as it showed lower ASR compared to $\sigma=2.0$ on SignHunter, NES, and ZoSignSGD*. | $\sigma$ | Accuracy Drop | Square ASR | | ---------- | :-----------: | :--------: | | 2.0 | ~0.4% | 66.11% | | 2.5 | ~0.6% | 70.94% | | 3.0 | ~0.7% | 82.39% | | 5.0 | ~4.0% | 100% | *Performance of various $\sigma$ values for WideResNet-28-10 against Square attack.* | $\sigma$ | Accuracy Drop | Square ASR | | ------------- | :-----------: | :--------------: | | 0.5 | ~0.3% | 74.23% | | **0.7** | ~0.7% | **71.75%** | | 1.0 | ~1.5% | 80% | | 1.4 | ~4.0% | 94.9% | **Q7: As some results in the paper are not convincing and evaluation settings are not comprehensive mentioned, I suggest the authors release your implementation for replication and verification purposes?** In our original submission, we included our implementation, which is available in the supplementary materials, which we believe may have escaped your attention. Please refer to the supplementary materials for the code. Additionally, we will expand on the evaluation settings in Section 5 and the Appendix of our paper to address any gaps and provide greater clarity. **Q8: In this study, EOT is employed as an adaptive attack and it can be incorporated with gradient-estimation or random search attack method. Although the authors use k queries for each sample at each attack iteration and take an average over the outputs to cancel out the randomness but it is unclear what method is used to determine attack direction e.g. gradient-based or random search approach. What is the perturbation budget used to attack against the adaptive attack? Again, the query average is very low if compared to the query budget but the ASR is not high. Can the author report the median query number instead and provide a ridiculous explanation?** The choice of attack method for EOT depends on the attack that incorporates EOT. In all the experiments in Table 5 we are using Square attack, which uses random search. Please refer to our response to **Q5** for the explanation on low query average when ASR is low. We will revise our Table 5 to make sure it is specifically stated that the adaptive attack uses Square attack.
Authors' Response to Reviewer wBXU (6/6)
**Q9: The authors discussed AAA defense which is designed for score-based attacks and this defense does not induce an accuracy drop as RLS. For completeness, I encourage the author to evaluate and compare this defense with RLS.** **Response:** We have discussed AAA in response to reviewer [ye5X](https://openreview.net/forum?id=mJzOHRSpSa¬eId=t0mVXZSzJx). We kindly refer you to [our reported AAA results](https://openreview.net/forum?id=mJzOHRSpSa¬eId=fPSLupXOqT) and [our adaptive attack against AAA](https://openreview.net/forum?id=mJzOHRSpSa¬eId=duzIbkJhhQ). **Q10: There are some contributions that are not actually contributions, e.g., 2 and 3. The authors should reorganize and summarize contributions 1 and 4 more thoroughly.** **Response:** Our contributions are as follows: 1. We propose a novel, simple, plug-and-play defense called RLS, which achieves state-of-the-art robustness against score-based adversarial attacks. This method maintains model accuracy without introducing additional overhead or requiring fine-tuning. Additionally, we provide a detailed analysis of how RLS preserves accuracy and disrupts the optimization strategies used by score-based attackers. 2. We introduce a straightforward adaptive attack against AAA to highlight its vulnerabilities. 3. We conduct extensive experiments on state-of-the-art randomized defenses, four models, and two benchmark datasets to demonstrate the robustness of RLS against score-based adversarial example attacks. Furthermore, we show that RLS is resilient to the EOT adaptive attack. We will revise the paper to improve the organization and clarity of the contributions. **Q11: Is minimizing distortion to the confidence score (in the introduction and section 5.3) really worth achieving? If so, the authors should cite papers or provide existing applications that rely on the accuracy of the gap between the model’s top and runner-up classes’ confidence scores to support your argument.** **Response:** In certain applications, the uncertainty associated with the model's output is a critical factor. One straightforward measure of this uncertainty is the gap between the top and runner-up class confidence scores. For example, in out-of-distribution detection [5], out-of-distribution samples typically exhibit a smaller gap. In medical applications [6], it is essential to rely on outputs with high confidence for decision-making. Since RLS directly influences the entire output probability vector of the model, Section 5.3 aims to evaluate and analyze how RLS impacts this gap, as well as its performance in comparison to other defense methods. **Q12: The experimental setup is not well explained and lacks some crucial information. For instance, what is the $\ell_p$ norm of the attacks? What is the perturbation budget used in this study? How many samples are selected for evaluation on CIFAR-10? How did the author select these samples? The author should clearly mention how samples are selected for evaluation on ImageNet in the experimental setup, e.g. 1000 samples are randomly selected, as mentioned in the caption of Table 4 in the experimental setup. Similarly, 2000 samples from CIFAR-10 are in the caption of Table 5.** **Response**: We will revise our submission to include detailed information about the experimental setup. The attacks in our paper use $\ell_\infty$ and the perturbation budget is 0.05. For experiments in Table 3 we use 1,000 randomly selected test samples from CIFAR-10. The 1,000 randomly selected ImageNet samples are drawn from the validation set and are the same samples used in RND [1]. [1] Zeyu Qin, Yanbo Fan, Hongyuan Zha, and Baoyuan Wu. Random noise defense against query-based black-box attacks. NeurIPS, 2021. [2] Nguyen Hung-Quang, Yingjie Lao, Tung Pham, Kok-Seng Wong, and Khoa D. Doan. Understanding the robustness of randomized feature defense against query-based adversarial attacks. ICLR, 2024 [3] Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square Attack: a query-efficient black-box adversarial attack via random search. ECCV, 2020 [4] Chen, Sizhe, et al. "Adversarial attack on attackers: Post-process to mitigate black-box score-based query attacks." Advances in Neural Information Processing Systems 35 (2022): 14929-14943. [5] Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training Confidence-Calibrated Classifiers For Detecting Out-Of-Distribution Samples. ICLR, 2018. [6] Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing The Reliability Of Out-Of-Distribution Image Detection In Neural Networks. ICLR, 2018.
Thank you for your responses to all my questions. However, many of them are not fully addressed. __Q1.__ Thank you for your response. I wonder if the empirical results apply to output softmax since I was confused with the class score and probability score. However, it is not the main problem now. It would be good if the authors could clarify the class score. The notation $f(x)$ denotes an output vector, and $f_i$ is a class score. Is it a probability score $p_i$ after softmax? __Q2/Q7. Theory and Implementation.__ The authors could not provide a theoretical analysis to support their proposed defense. The author could provide a general theory to connect random logit scaling to query expense or robustness. Additionally, I have checked the submitted code, but it seems that the instruction (README file) was not well-written and the defense implementation was missing so I could not test it. It would be good if the authors could provide comprehensive and straightforward instructions to reproduce your proposed defense. The empirical results look impressive, but it is hard to assess them without theoretical analysis or reproducible code. __Q3. Robustness evaluation against $l_0$.__ The authors did not provide results to demonstrate the robustness of RLS against $l_0$ attacks. __Q4. Results for Bandits__ If I'm not mistaken, the ASR of Bandits at high perturbation (0.1) is lower than at low perturbation (0.05) if using RLS (0.5, 1000). I think the authors should verify these results. __Q5. Average number of queries.__ The explanation should be more straightforward and concise. How do you compute the average query number? Is it based on successful attacks only (ignore failed attacks)? What are "hard samples"? How do you quantify it, and how many percentages do they account for? It could affect the performance of different defenses. __Q6. The choice of $\sigma$, RFD.__ I think the higher $\sigma$ is, the more robust RFD is. Then, the ASR should be low. However, it seems the results in the response are not my expectation. If I'm not correct, can the authors provide an explanation? Additionally, my main point is that the authors should choose the hyperparameters where RFD has 1% or 2% drop clean accuracy. Because, I think the authors of RFD aimed to select hyperparameters such that RFD obtained 1% or 2% drop for all their experiments. In your study, hyperparameters of RFD are not consistent, and I do not know why you chose them.
Authors' Response to Reviewer wBXU (1/3)
We thank the reviewer for their valuable feedback and are pleased that our previous response addressed some of your concerns (from **Q8** to **Q12**). We have carefully considered your additional feedback prior to the rebuttal deadline and have already addressed some of the concerns in our rebuttal revision. We hope to address your remaining concerns. **Q1. Thank you for your response. I wonder if the empirical results apply to output softmax since I was confused with class score and probability score. However, it is not the main problem. It would be good if the authors could clarify what is the class score. The notation $f(x)$ denotes an output vector and $f_i$ is a class score. Is it a probability score $p_i$ after softmax?** **Response**: We would like to clarify that we use the term "logits" to refer to the unnormalized outputs of the model (prior to the softmax layer), while we use the terms "confidence score," "class score," and "probability score" to refer to the probabilities obtained after the softmax layer. --- **Q2/Q7. Theory and Implementation. The authors could not provide a theoretical analysis to support their proposed defense. The author could provide a general theory to connect random logit scaling to query expense or robustness. Additionally, I have checked the submitted code, but it seems that the instruction (README file) was not well-written and the defense implementation was missing so I could not test it. It would be good if the authors could provide comprehensive and straightforward instructions to reproduce your proposed defense. The empirical results look impressive, but it is hard to assess them without theoretical analysis or reproducible code.** **Response:** > The authors could not provide a theoretical analysis to support their proposed defense. The author could provide a general theory to connect random logit scaling to query expense or robustness. As mentioned in our previous response, we did not provide a theoretical analysis of the non-smooth characteristics of RLS since it required resorting to oversimplification or overapproximation. To maintain accuracy, RLS relies on a strictly positive distribution. Specifically, we use a uniform distribution on $(a, b)$, where $0 < a < b$. On other distributions such as a Gaussian distribution, RLS necessitates modifications to ensure that only positive values are sampled, thereby effectively disrupting the smooth properties of the distribution. To solve this issue, we have now provided a comprehensive set of empirical evaluations of RLS using six SOTA attacks across various $\ell_p$ norm spaces ($\ell_0$, $\ell_2$, and $\ell_\infty$) for both ImageNet and CIFAR-10 in all of the norm spaces. These evaluations consistently demonstrate that RLS outperforms other defenses. We also provided an analysis regarding how logit scaling affects attackers that use black-box score-based optimizations in our original submission. > the defense implementation was missing so I could not test it **We would like to clarify that defense implementations are indeed included**. In our original submission, we provided implementations of RFD, oRND, iRND, and our method, RLS. Additionally, in response to the reviewers' feedback during the discussion phase, we have included the AAA defense in the final revision for comparison. We have also added the implementation of our adaptive attack against AAA. > Additionally, I have checked the submitted code, but it seems that the instruction (README file) was not well-written. It would be good if the authors could provide comprehensive and straightforward instructions to reproduce your proposed defense. We have now expanded the instructions to make them easier to follow. Please find the updated README file in the supplementary material of our submission. --- **Q3:** **Robustness evaluation against $\ell_0$. The authors did not provide results to demonstrate the robustness of RLS against l0 attacks.** **Response:** Initially, we followed prior works (RFD, oRND, iRND, AAA) that do not evaluate in the $\ell_0$ norm space. However, **in our revised submission, we have now included evaluations in the $\ell_0$ space. Specifically, we used BruSLeAttack [1], the SOTA sparse black-box score-based attack**, under various sparsity levels against ResNet-50 (ImageNet) and ResNet-18 (CIFAR-10). The results, which are **reported in Appendix F.2**, confirm that RLS outperforms other defenses in this configuration as well.
Authors' Response to Reviewer wBXU (2/3)
**Q4: If I'm not mistaken, the ASR of Bandits at high perturbation (0.1) is lower than at low perturbation (0.05) if using RLS (0.5, 1000). I think the authors should verify these results.** **We would like to clarify that it is inaccurate to state that larger perturbations always result in a higher ASR. This claim is contradicted by similar evaluations in the related work.** For instance, in Table 8 of iRND [2] (evaluations conducted on VGG-16), the failure rate for $\epsilon = 0.01$ increases by 14.5% compared to $\epsilon = 0.001$ for ZO-signSGD in $\ell_2$ (for $\nu = 0.00$). In the same table, the failure rate of Bandit ($\ell_2$) increases by 0.4% for $\epsilon = 0.01$ compared to $\epsilon = 0.001$ (both for $\nu = 0.01$). Additionally, Table 3 of RFD [3] shows that the robustness of both VGG and ViT consistently increases (ASR decreases) against Square attack as $\epsilon$ is increased from 0.05 to 0.1. Based on the same table, NES Attack's ASR on VGG decreases for $\epsilon = 0.2$ compared to $\epsilon = 0.01$. In general, the effect of larger perturbations on the ASR depends on the defense, target model architecture, and attack method employed. Furthermore, **the difference between the two cases highlighted by the reviewer is approximately 1% (specifically 1.02% and 0.97%), occurring at the very lower bound of the ASR, where the ASR is effectively reduced to nearly 0%**. We believe this small difference is not particularly significant, as RLS reduces the success rate to around 0% in both cases, and randomized defenses can introduce slight variations in the results, thereby accounting for the small difference observed between the two cases. --- **Q5 Average number of queries. The explanation should be more straightforward and concise. How do you compute the average query number? Is it based on successful attacks only (ignore failed attacks)? What are "hard samples"? How do you quantify it, and how many percentages do they account for? It could affect the performance of different defenses.** **Response**: > How do you compute the average query number? Is it based on successful attacks only (ignore failed attacks)? In line with previous works on attacks and defenses, we are reporting the average number of queries for successfully crafted samples. We would like to kindly clarify that **we have also explicitly stated this in our original submission in the titles of Table 3 and Table 4, where we used the phrasing "The average number of queries for successfully crafted adversarial samples."** > What are "hard samples"? How do you quantify it, and how many percentages do they account for? It could affect the performance of different defenses. We apologize for any confusion caused by our previous explanation and regret that it did not address your concern. To clarify our response, we will rephrase it more clearly and concisely: The term "hard" samples was used figuratively to describe samples that exhaust the query budget but remain non-adversarial (i.e., adversarial attacks fail to transform them into adversarial examples, in other words the predicted label by the victim model for the samples remains unchanged). We did not intend to introduce a new concept. The key point is that when the ASR drops significantly, such as in the case of Bandit with RLS (0.5, 1000), the majority of samples remain non-adversarial throughout the query budget, while only a small minority are successfully transformed into adversarial examples. For instance, in the case of Bandit against RLS (0.5, 10), with an ASR of less than 15% and an average query count of approximately 60 (compared to around 300 queries for the undefended model), **the low average number of queries is due to the fact that over 97% of the successfully crafted adversarial examples require 100 queries or fewer (as we reported in our previous response [in this post](https://openreview.net/forum?id=mJzOHRSpSa¬eId=oJvnDZur0F)). Consequently, the remaining ~85% of the samples exhaust the query budget without success, which results in a low average number of queries for successfully crafted samples**. In contrast, for the undefended model, the Bandit attack successfully crafts adversarial examples for all 100% of the samples, and approximately 50% of these require more than 100 queries, thereby increasing the average number of queries for successfully crafted samples.
Authors' Response to Reviewer wBXU (3/3)
**Q6: **The choice of** $\sigma$ **RFD.** I think the higher $\sigma$ is the more robust RFD is. Then, the ASR should be low. However, it seems the results in the response are not my expectation. If I'm not correct, can the authors provide an explanation? Additionally, my main point is that the authors should choose the hyperparameters where RFD has 1% or 2% drop clean accuracy. Because, I think the authors of RFD aimed to select hyperparameters such that RFD obtained 1% or 2% drop for all their experiments. In your study, hyperparameters of RFD are not consistent, and I do not know why you chose them.** **Response**: > I think the higher $\sigma$ is the more robust RFD is. Then, the ASR should be low. However, it seems the results in the response are not my expectation. If I'm not correct, can the authors provide an explanation? **We clarify that higher $\sigma$ does not always lead to increased robustness. This is confirmed by the evaluations in the RFD [3] paper.** For instance: - **Table 1 in [3]** shows that the robustness of RFD against both Square and SignHunter drops by 0.4% for the case with a larger $\sigma$ (on DeiT). In the same table, the robustness of VGG-16 decreases against SignHunter for larger $\sigma$. - **Table 2 in [3]** further demonstrates that the robustness drops by ~4% against NES (for ResNet-50) when using a larger $\sigma$. In fact, the robustness against NES consistently drops for larger $\sigma$ across all models used (VGG-19, ResNet-50, DeiT, and ViT). SignHunter shows a similar situation against DeiT. The robustness alos decreases by ~2% against Square (on DeiT). - **Table 3 in [3]** indicates that the robustness of VGG against SignHunter decreases by ~3% for larger $\sigma$ (for $\epsilon = 0.01$). Additionally, the robustness against NES decreases consistently for larger $\sigma$ on both VGG and ViT. The same table also shows that the robustness of both VGG and ViT decreases against Square for larger $\sigma$ values in some cases, with reductions ranging from 0.3% to 1.5%. **To further address the reviewer's concerns, we kindly refer you to Table 12 of RFD [3] in Appendix E.2, which explicitly states that their method is weaker than AAA for score-based attacks. In contrast, we demonstrated that RLS outperforms AAA. Consequently, we firmly believe that *there are no configurations of RFD that can outperform RLS*.** > Additionally, my main point is that the authors should choose the hyperparameters where RFD has 1% or 2% drop clean accuracy. Because, I think the authors of RFD aimed to select hyperparameters such that RFD obtained 1% or 2% drop for all their experiments. In your study, hyperparameters of RFD are not consistent, and I do not know why you chose them. **We would like to further emphasize and clarify that, for a fair comparison between the defenses, we experimented with a range of $\sigma$ values for each model and selected the one that performed best for the majority of attacks, as stated in our previous response. To be more specific, *the criterion for choosing the $\sigma$ values was to identify those that resulted in accuracy drops of 0% to 4% and select the one that worked best across most attacks*.** The ~1% or ~2% accuracy drops reported in [3] represent the average accuracy drops over multiple executions. The accuracy drops vary across models for different $\sigma$ values; therefore, the $\sigma$ values used in our evaluations are not uniform. For example, in our evaluations we observed that VGG-16's accuracy drops by ~2% for a small $\sigma = 0.35$, whereas ResNet-18's accuracy drops ~1% with a much larger $\sigma = 2.5$. **We reiterate that [3] explicitly states that they cannot outperform AAA, whereas RLS does. This confirms that regardless of the choice of $\sigma$, *RFD will never outperform RLS*.** [1] Quoc Viet Vo, Ehsan Abbasnejad, and Damith C. Ranasinghe. Brusleattack: Query-efficient score-based sparse adversarial attack. ICLR, 2024. [2] Zeyu Qin, Yanbo Fan, Hongyuan Zha, and Baoyuan Wu. Random Noise Defense Against Query-Based Black-Box Attacks. NeurIPS, 2021. [3] Nguyen Hung-Quang, Yingjie Lao, Tung Pham, Kok-Seng Wong, and Khoa D. Doan. Understanding the robustness of randomized feature defense against query-based adversarial attacks. ICLR, 2024.
Dear Reviewer wBXU, We thank you again for your detailed review and feedback. As the discussion period is nearing its end, we would like to know whether the revised version of our paper and our responses have addressed your concerns. We would be happy to discuss any remaining concerns and provide further clarification.
Summary
Existing works show that randomizing the prediction mitigates the threat of query-based attacks. However, defenses that inject noise into the input or to the feature lead to accuracy drops on clean data. Instead, this paper proposes random logit scaling (RLS), a defense that rescales the returned logit by a random variable $m$, therefore remains clean accuracy. Experimental results show that RLS is robust to score-based black box attacks without the tradeoff in accuracy.
Strengths
- RLS is simple and plug-and-play, and can be applied to any model. - RLS is robust against score-based attacks and does not change the accuracy of the target model. - The experiments show that RLS is also robust against adaptive attacks that query the victim multiple times and average the prediction.
Weaknesses
- Since RLS does not change the prediction, it will not be effective against decision-based attacks. The argument that decision-based attacks are inefficient is not convincing; recent attacks like RayS [1] can find adversarial samples with ~100 queries. - The paper should compare to AAA [2], another defense that changes the logit to fool the attacker. AAA also guarantees the reliability of the returned probability by optimizing the calibration error. - The experiments are only conducted on CIFAR10, which is insufficient to conclude the effectiveness of RLS. [1] Chen, Jinghui, and Quanquan Gu. "Rays: A ray searching method for hard-label adversarial attack." Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020. [2] Chen, Sizhe, et al. "Adversarial attack on attackers: Post-process to mitigate black-box score-based query attacks." Advances in Neural Information Processing Systems 35 (2022): 14929-14943.
Questions
- What is the performance of AAA? - What is the calibration error of RLS? - Experimental results on other datasets such as ImageNet would be helpful to demonstrate the robustness of RLS.
Rating
5
Confidence
5
Soundness
2
Presentation
2
Contribution
2
Authors' Response to Reviewer ye5X (1/3)
We thank you for your review and valuable insights and hope to address your concerns and questions thoroughly in this response. ## Weaknesses **W1**: Rays [1], while one of the most efficient decision-based attacks, is still dramatically less efficient than SOTA score-based attacks. For instance, Square attack [2], a SOTA score-based attack, **achieves 100% success rate on both CIFAR-10 and ImageNet with an average of only ~100 and ~70 queries, respectively**. In contrast, **Tables 2 and 3 of [1] reported by RayS authors indicate that RayS requires an average of 792.8 and 574.0 queries for CIFAR-10 and ImageNet, respectively, and that RayS does not find adversarial queries with ~100 queries**. This demonstrates that square attack is at least 5x more efficient than RayS, achieving successful attacks with dozens of queries rather than hundreds for both datasets. Therefore, we contend that the effectiveness of decision-based attacks is still significantly lagging behind score-based attacks. Also, it is worth noting that AAA is also only effective against score-based attacks. **W2:** Addressed in response to questions 1 and 2. **W3:** Addressed in response to question 3. ## Questions Below are our responses to your questions: **Q1: What is the performance of AAA?** **Response:** We have now evaluated the performance of AAA against SignHunter [3] and Square [2] attack, two of the most effective attacks with the highest ASR and lowest query counts as reported in Tables 3 and 4 of our work and compared it to that of RLS as reported in the table below. The evaluation was conducted using the same query budget and samples as specified in our original submission, and we use the original settings provided by the authors of AAA [4] for evaluations on AAA-sine (we do not use AAA-linear as it has been shown to be vulnerable to a straightforward adaptive attack, as detailed in [the discussion here](https://openreview.net/forum?id=7hhH95QKKDX¬eId=-NwL_TnNb8QC)): *ASR and average number of queries for Square and SignHunter attacks against ResNet-18 with defense methods AAA-sine and RLS* | Model/Dataset | Defense | Square | SignHunter | | :-------------------------: | :-------------: | --------------------------- | ---------------------------- | | ResNet-18 / CIFAR-10 | AAA-sine | 45.80% / 114.51 | 32.61% / 242.61 | | | RLS (0.5, 1000) | **43.05% / 92.10** | **17.85% / 978.58** | | VGG-16 / CIFAR-10 | AAA-sine | 27.05% / 108.19 | 31.25% / 264.66 | | | RLS (0.5, 1000) | **21.56% / 38.58** | **15.85% / 224.44** | | WideResNet-28-10 / CIFAR-10 | AAA-sine | 46.40% / 120.87 | 34.10% / 436.00 | | | RLS (0.5, 1000) | **41.38% / 127.47** | **15.08% / 1168.73** | | ResNet-50 / ImageNet | AAA-sine | 57.20% / 103.66 | 41.00% / 208.26 | | | RLS (0.5, 1000) | **48.52% / 13.76** | **21.41% / 104.46** | The results indicate that RLS achieves a higher robustness than AAA-sine. We would also like to provide a more detailed comparison between our work and AAA, as our original submission primarily focused on randomization-based defenses, similar to our method, RLS:
Authors' Response to Reviewer ye5X (2/3)
**AAA is Vulnerable to Adaptive Attackers:** An inherent limitation of AAA is that it transforms the true loss function into another pre-defined function, such as a sine or linear function with a slope opposite to that of the true loss values. This transformation makes AAA vulnerable to adaptive attacks. Given that an attacker can reasonably assume the presence of AAA (since relying solely on security through obscurity is generally considered flawed), they can leverage effective adaptive techniques against this defense. In contrast, randomization-based defenses, such as RLS, do not have any known effective adaptive attacks, with the exception of EOT, which we showed to be very ineffective against RLS (as detailed in Table 5 in Section 5.5 of our paper). The authors of the original AAA paper discussed adaptive attacks against AAA-linear (as detailed in [the discussion here](https://openreview.net/forum?id=7hhH95QKKDX¬eId=-NwL_TnNb8QC)), but we demonstrate that AAA-sine is also susceptible to adaptive attacks. Following the original work [4] on an adaptive attack against AAA-linear using Square attack, we develop and implement a simple adaptive attack against AAA-sine and implement it on Square attack that can increase the ASR by ~36% and achieving up to ~82% ASR. AAA-sine can be viewed as a smoothed version of AAA-linear: it divides the true loss into intervals and transforms the logits such that the resulting loss takes on a sine-like shape, causing the loss to sometimes point in the opposite direction of the true loss gradient within certain parts of each interval. Both random-search and gradient-based attacks generally follow the direction that minimizes the loss, hence, AAA-sine misleads attackers by reversing the gradient direction in some parts of the loss curve, effectively trapping them in local minima. In other words, the attacker is drawn into a local minima of the sine-like loss curve where they are unable to identify the correct direction for minimizing the true loss, since the proper direction is opposite to the gradient provided by AAA-sine. Our adaptive attack changes the direction that the attacker follows: when the attacker fails to find a direction that reduces the loss for $k$ attack iterations, they shift to the opposite direction, one that increases the loss. After several iterations however, the attacker may become stuck at a local maximum of the sine-like loss curve. To overcome this, after another $k$ iterations with no success in moving up the loss curve, the attacker again changes direction. By iteratively switching directions upon $k$ consecutive attack iterations with no success in finding the desired direction, the attacker will be able to climb up the local minima in the loss curve that AAA-sine generates and climb back down from local maxima again when it fails to find an upward direction for $k$ attack iterations. We report ASR and the average number of queries for our adaptive attack for values of $k=5$ and $k=10$ against AAA-sine with the configuration of the original AAA defense. Our adaptive attack is referred to as "Pr-Square-$k$". We evaluate its performance using the same ResNet-18 model and the same randomly selected 1,000 test samples from CIFAR-10 that were used in our experiments to assess the performance of the defense methods: | Attack | AAA-sine | | :----------: | :-------------------------: | | Square | 45.80% / 114.51 | | Pr-Square-5 | 74.88% (+29.08) / 2369.62 | | Pr-Square-10 | 81.42% (+35.62%) / 2946.09 | **AAA Introduces Additional Runtime Overhead** Another limitation of AAA is that its logit optimization post-processing module introduces additional overhead for model predictions. This added computational cost can be problematic in critical scenarios that require fast, real-time responses from AI systems, such as in self-driving cars. To assess the runtime overhead of AAA-sine logit optimization, we used a NVIDIA Tesla T4 GPU with 16 GB of memory and followed the original configuration provided by the authors of AAA, which includes 100 iterations of optimization. Our results show that AAA introduces a minimum runtime overhead of 0.13 seconds and an average overhead of 0.15 seconds. While this delay may appear small in isolation, it can be dangerous in time-sensitive applications, such as self-driving cars operating at high speeds on highways. In contrast, RLS does not introduce any measurable runtime overhead. **We will revise our submission to include a detailed comparison of AAA and RLS, highlighting AAA's weaknesses relative to RLS, including our contribution of presenting a novel adaptive attack against AAA-sine, as well as discussing the additional overhead associated with AAA.**
Authors' Response to Reviewer ye5X (3/3)
**Q2: What is the calibration error of RLS?** **Response:** Below, we report the calibration error of RLS. To evaluate this, we compute the minimum, average, and maximum ECE for each defense method using bins of size 100, feeding all 10,000 CIFAR-10 test samples through ResNet-18 five times. Additionally, we would like to highlight that we have also examined the skewed probabilities introduced by our method, as well as other randomized state-of-the-art defenses, in terms of "confidence score distortion" on ResNet-18 (detailed in Section 5.3 of the paper). *Minimum, average, and maximum ECE of ResNet-18 for each defense under the 10,000 test samples of CIFAR-10. Five executions are used for taking the average*. | Defense | Min | Avg | Max | | ----------------------- | :---------------: | :---------------: | :---------------: | | No Defense | 0.0268 | - | - | | AAA-sine | 0.0220 (-0.0048) | 0.0220 (-0.0048) | 0.0220 (-0.0048) | | RLS (0.5, 10) | 0.0331 (+0.0063) | 0.0401 (+0.0133) | 0.0429 (+0.0161) | | RLS(0.5, 100) | 0.0447 (+0.0179) | 0.0448 (+0.0180) | 0.0448 (+0.0180) | | RLS(0.5, 1000) | 0.0447 (+0.0179) | 0.0448 (+0.0180) | 0.0448 (+0.0180) | | iRND ($\nu$ = 0.01) | 0.0284 (+0.0016) | 0.0302 (+0.0034) | 0.0319 (+0.0051) | | iRND ($\nu = $ 0.02) | 0.0439 (+0.0171) | 0.0454 +(0.0186) | 0.0478 (+0.0210) | | RFD ($\sigma$ = 2.5) | 0.0261 (-0.0007) | 0.0267 -(0.0001) | 0.0279 (+0.0009) | | oRND ($c$ = 1) | 0.0257 (-0.0011) | 0.0263 (-0.0005) | 0.0282 +(0.0014) | The results show that RLS increases the ECE by a range of $0.0063$ to $0.0180$, while iRND leads to an increase between $0.0016$ and $0.0021$. Other defenses exhibit minimal impact on ECE. However, given the superior robustness of RLS against adversarial attacks, we consider the small increase in ECE to be acceptable. Moreover, RLS preserves model accuracy, unlike iRND, RFD, and oRND, and offers greater robustness against adaptive attacks compared to these defenses. Additionally, RLS does not incur the extra computational overhead for logit optimization, as seen in AAA-sine. Moreover, as discussed in **Q1**, RLS outperforms AAA-sine on SignHunter and Square and AAA-sine is susceptible to simple adaptive attacks. Furthermore, RLS requires minimal fine-tuning across different models and attacks, in contrast to RFD, which necessitates the fine-tuning of a $\sigma$ value for each model the defender aims to protect. **Q3: Experimental results on other datasets such as ImageNet would be helpful to demonstrate the robustness of RLS.** **Response**: In our original submission, we presented experimental results on ImageNet using ResNet50, which we believe may have escaped your attention. We kindly refer you to Section 5.4 and Table 4 for experimental results on ImageNet. [1] Chen, Jinghui, and Quanquan Gu. "Rays: A ray searching method for hard-label adversarial attack." Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020. [2] Lin, Maximilian, et al. "Square attack: A query-efficient black-box adversarial attack via random search." Proceedings of the European Conference on Computer Vision (ECCV). 2020. [3] Al-Dujaili, Abdullah, et al. "SignHunter: Sign bits are all you need for black-box attacks." Proceedings of the Advances in Neural Information Processing Systems (NeurIPS). 2020. [4] Chen, Sizhe, et al. "Adversarial attack on attackers: Post-process to mitigate black-box score-based query attacks." Advances in Neural Information Processing Systems 35 (2022): 14929-14943.
Thank you for your informative response. It'd be helpful if you can include the discussion about AAA in the revision. I still have the following concern - RLS induces very high calibration errors. More specifically, RLS(0.5, 1000), which is the effective configuration in the experiments, has errors similar to those of iRND with large input noise. It's hard to conclude that RLS returns a reliable probablity. - Is there any explanation for why RLS is more robust than previous methods such as RND, RFD, and AAA? In general, most of my initial questions are addressed. I adjust the rating accordingly.
Authors' Response to Reviewer ye5X
We thank you for your response, your suggestions, and for adjusting your score. In our revision, we will include our robustness evaluations on AAA, the ECE evaluations, and also our novel contribution of proposing an effective adaptive attack against AAA. In the following, we hope to address your remaining concerns: **Q1: RLS induces very high calibration errors. More specifically, RLS(0.5, 1000), which is the effective configuration in the experiments, has errors similar to those of iRND with large input noise. It's hard to conclude that RLS returns a reliable probablity.** **Response:** While it is true that RLS (0.5, 1000) increases ECE by 1.8%, which is slighlty lower than iRND with $\nu = 0.02$, we would like to highlight that RLS (0.5, 10) still outperforms all the other randomized defenses while increasing ECE by only 1.3% on average. Additionally, the randomized nature of RLS makes it more challenging for attackers to carry out effective adaptive attacks, in contrast to AAA, against which we proposed an effective and straightforward adaptive attack. Furthermore, we would like to highlight that RLS outputs remain highly reliable in two key aspects: 1. The top class remains the same ensuring that accuracy does not decrease and there is no increase in false predictions, unlike other randomized defenses. 2. Interestingly, our evaluations demonstrate that the distortion of output probabilities caused by RLS is comparable to other randomized defenses and even minimizes it for RLS (0.5, 10), as detailed in Table 2 and Section 5.3 of our original submission. We attribute this to the fact that, for samples drawn from the model's expected data distribution the model is expected to assign very high confidence scores to the correct class. Our evaluations on CIFAR-10 (for the 1,000 test samples with the highest top-class confidence scores among the 10,000 test samples) show that **for $m=0.5$, the confidence score of the correct class decreases by less than 5%, while for $m=10$, the confidence score increases by only 0.02%**. We further emphasize that every defense comes with its costs. iRND, oRND, and RFD reduce model accuracy while showing a significantly lower robustness compared to RLS. Additionally, RFD requires extra fine-tuning of its parameter $\sigma$ for each defended model, whereas RLS offers superior robustness in both configurations across all models and attacks. AAA introduces additional overhead and is vulnerable to adaptive attacks. On the other hand, RLS may be problematic in applications that are highly sensitive to calibration errors. **Q2: Is there any explanation for why RLS is more robust than previous methods such as RND, RFD, and AAA?** **Response:** RND and RFD are based on the idea that when a defender adds random noise to input or intermediate output features, it disrupts the noise $\mu u$ that attackers introduce to the input data in order to estimate the gradient (gradient-based) or search for potential perturbations (search-based). However, even with the noise introduced by the defender, the attacker may still be able to obtain a reasonable estimate of the gradient or identify effective directions in its search, potentially by chance, especially in search-based methods. However, RLS directly falsifies the output scores, turning the attackers' computed loss function into a highly non-smooth curve (as shown in Figure 2a of our original submission). During each inference, the shape of the loss function changes (as each time a different value of $m$ is sampled). Consequently, RLS can be viewed as an ensemble of infinitely many models each producing its unique attack loss function shape (corresponding to each possible value of m). This variability makes it extremely difficult for any attacker to navigate the loss curve whether through search-based methods or gradient estimation. Similarly, AAA manipulates the output scores and transforms them such that the attacker's objective loss function takes the shape of a linear curve opposing the true loss or a sine-like wave. Both scenarios are vulnerable to adaptive attacks and show greater smoothness compared to the loss shapes produced by RLS. We attribute the relative underperformance of AAA compared to RLS to the smoother loss curve characteristic of AAA. Additionally, AAA still guides the attacker in the right direction in some intervals of its sine-shaped output.
Thank you for your clarification. I'd like to discuss more on those points. **Q1:** I believe that a reliable, not only high, confidence score is necessary. This means the model returns high scores for samples with a high chance of being correct and low scores for other cases. Another effective defense is to return a fixed high probability, say 95%, for every sample. I suppose that there is no black-box score-based attack or any adaptive strategy that can beat this defense, but a fixed probability is not reliable at all. **Q2:** Intuitively, the noise injected by RND or RFD also has the same effect of turning the loss into a non-smooth function due to the randomness. We can also view RND and RFD as ensembling, where each member is associated with a noise sample. It's not clear to me why RLS is more effective in general, except for the experimental results.
Authors' Response to Reviewer ye5X
We thank you again for the follow-up. We hope to address your concerns in the following. **Response to Q1**: While it is true that RLS can increase the calibration error of the victim model (for $a = 0.5$ and $b = 10$, the average calibration error increases by 1.3%, and for $a = 0.5$ and $b = 1000$, the average increase is 1.8%), the effects on confidence scores remain relatively modest. We have more data to back up this claim. We analyze the **extreme case where $m = 10$** (with RLS parameters $a = 0.5$, $b = 10$) using 10,000 CIFAR-10 test samples on ResNet-18, including both correctly classified and misclassified instances. In this scenario (for $m=10$, the top-class confidence score increases while the runner-up confidence score decreases): - **92% of the samples experience an increase of less than 5%** in the confidence score of the predicted top class. - **93% of the samples see a less than 5% decrease** in the confidence score of the runner-up class. - Confidence score changes in lower-ranked classes are even smaller. At the **other extreme, for $m = 0.5$** (the top-class confidence score decreases while the runner-up confidence score increases): - Approximately **90% of the samples experience less than a 5% increase** in the confidence score of the runner-up class. - The confidence score of the top class **decreases by less than 10%** for approximately **76%** of the samples. - Confidence score changes in lower-ranked classes are even smaller. - Furthermore, **among the most confident samples**, those with a **top-class confidence score of 90% or higher** (approximately **94%** of the 10,000 samples), **~79% will retain a probability of 90% or higher**, and **~93% will retain a probability of 80% or higher**. Additionally, **~96% of the samples will still have a runner-up class with a confidence score below 10%**. It is important to note that these numbers represent extreme cases. In practice, RLS samples $m$ values uniformly, resulting in generally milder effects on confidence scores. Additionally, RLS can be augmented with an optimization module which adjusts the scaled logits to reduce calibration error, similar to AAA. This is particularly helpful for applications that require highly calibrated outputs. However, it should be noted that this optimization would come at the cost of an output delay, similar to the trade-offs with AAA. RLS can also be adjusted with milder $a$ and $b$ values to have even smaller effects on the output probability vector. --- **Response to Q2**: We clarify that RLS directly disrupts and reshapes the objective function by applying transformations for each $m$ value that modify the output logits and consequently affect the attacker's objective function. This *targeted* approach ensures that RLS actively interferes with the attacker's optimization process and objective function, making it more challenging to approximate gradients or compute meaningful updates. Similarly, AAA has a similar effect by directly altering output logits, thereby guaranteeing significant interference in the attacker's computations. In contrast, methods such as RND and RFD, which introduce random noise to the model's input or intermediate outputs, do not provide the same level of assurance. RND and RFD add noise in a more *aimless* manner, hoping to introduce difficulty into the attacker's optimization process. However, due to the inherent non-linearity and complexity of defended models, the random perturbations introduced by RND and RFD may not consistently result in meaningful changes to the model's final output. As a result, the attacker's objective function might remain largely unaffected in some cases, allowing the adversary to bypass these defenses more effectively. Both RLS and AAA ensure a more impactful effect on the attacker's optimization process by directly targeting the logits. However, the randomized nature of RLS makes it more difficult for attackers to adaptively attack our defense, unlike AAA, which we have shown to fail significantly against adaptive attackers.
Thank you for your detailed response. I appreciate the discussion on the calibration of RLS, however, I'd like you to clarify more on the effectiveness of RLS compared to other randomized defenses. You said *"RLS actively interferes with the attacker's optimization process and objective function'"*, however, as far as I understand, RLS applies a random temperature on the logits that is independent of the attack. I don't see a clear distinction between that and the noise added in RND/RFD. What are the meaningful changes that RLS induces but RND and RFD do not? For example, the analysis in RFD shows that the effectiveness of injecting noise into hidden layers comes from the ratio of the gradient norm, an explanation that I believe RLS is lacking.
We thank you for your review and hope to have addressed your concerns regarding the *reliability of RLS outputs* and the *explanation behind our method's advantages* over other defenses in our previous response. Additionally, in order to address your initial concerns regarding *comparisons with AAA* and *the calibration error*, in our revised version we provided a more detailed discussion on AAA in Section 2.2, proposed an adaptive attack against AAA in Section 4 (with the results presented in Section 6.2), compared the robustness of RLS with AAA in Appendix F.4, and presented the results of the defenses' expected calibration errors in Section 6.3 and Table 6.
We appreciate and thank you for your follow-up and are pleased that you found our discussion on the calibration of RLS insightful. RLS and AAA directly influence the loss trend utilized by attackers (as defined in equations (1) and (2) of our paper). In the case of random search attacks, where the attackers greedily choose directions that minimize the loss, AAA deceives the attackers by outputting a falsely higher score (lower score) for a perturbation that actually results in a lower true loss (higher true loss), due to the sine-like transformation that essentially oscillates the output loss curve around the true loss curve. RLS misleads attackers in a similar manner, by producing a higher score for the predicted class when a perturbation actually reduces the score and, consequently, the loss (logits scaled by $m > 1$); and also, by outputting a lower score for the predicted class for an incorrect action that increases the loss (logits scaled by $m < 1$). This effect cannot be achieved merely by adding noise to the model input or its intermediate features, as there is no guarantee that the noise will induce sufficiently large changes in the output scores to mislead the attacker, **nor that such changes will reverse the direction of the search**. We would also like to highlight that the randomized nature of RLS makes it more difficult for attackers to adaptively attack our defense compared to AAA, which we showed to be vulnerable against adaptive attackers. The same applies to gradient-based attacks. These attacks essentially attempt to recreate the loss curve around the current adversarial example $x_{\text{adv}}$ and compute the gradient by feeding slightly perturbed samples to the model and calculating the loss based on the resulting scores. AAA reshapes the local loss by introducing its sine-like transformation, which causes the attacker to ultimately encounter part of the transformed loss surface instead of the true loss. RLS is even more disruptive, transforming the otherwise smooth loss surface into a highly non-linear one. This is achieved by sampling a different $m$ for each minor perturbation, ensuring either a lower or higher loss computed by the attacker. In contrast, noise introduced by RFD and RND merely attempts to alter the loss for the perturbed samples but can have minimal impact on the output **since there is no straightforward relationship between the input to the model (or any of its layers) and the resulting changes in the model’s output**. Post-processing defenses, such as RLS and AAA, provide more direct control over the output scores, and consequently, the loss utilized by attackers, resulting in a significantly greater impact on their effectiveness.
I respectfully disagree with the argument that RLS guarantees that it induces the changes that reverse the direction of the search. For example, if the attack proposes a true direction that leads to adversarial samples, there will still be a chance that RLS updates the logit with m < 1, which further decreases the probability of the groundtruth label and emphasizes the update, and vice versa. In that case, RLS fails to mislead the attacker. If there is anything I misunderstand in the method and the analysis, please feel free to correct me.
We appreciate the reviewer’s follow-up and thank them for requesting further clarification. We clarify that we did not claim RLS ensures an opposite direction but emphasized that, because RLS directly modifies output logits, it consistently causes significant changes to the loss computed by the attacker. We also would like to clarify that this is not a limitation of RLS since AAA can also point in the correct adversarial direction within certain intervals of its sine-like loss that align with the flow of the true loss. Please refer to Sections 5.1, 5.2.2, and Appendices B and C of our paper, which detail how RLS affects the loss. Additionally, Figure 2a illustrates the loss changes induced by RLS. Post-processing defenses like AAA and RLS directly alter the loss landscape and remain unaffected by input or model non-linearities. In contrast, random noise applied to inputs or features does not result in consistent loss changes due to the model's non-linearities and complexity. In other words, the random noise in RND and RFD fails to consistently affect the loss landscape and while it can distort gradients, it does not reliably impair attacks. The weaker performance of other randomized defenses in disrupting attack algorithms, compared to post-processing defenses such as RLS, is empirically evident. To illustrate this, we conduct an experiment using the Square attack on an undefended ResNet-18 model, utilizing the same 1,000 CIFAR-10 test samples as in our robustness evaluations. In this setup, the attacker observes the true loss and crafts its queries and perturbations against the undefended model. However, for each attacker query, we also compute two additional adversarial losses: one with random noise added to the input for RND ($v=0.02$) and another with logit scaling for RLS (0.5,10). In other words, while the attacker targets the undefended model and observes the true loss, we simultaneously compute the loss under each respective defense for every query made by the attacker. We then compare the direction in which the true loss guides the attacker with the direction that RND and RLS losses would have guided them if the defenses were active. Specifically, we measure the alignment between the attacker's true loss direction and the respective defensive losses' directions. We assume the true loss direction is the optimal direction and classify any deviation caused by the defensive loss as a misdirection introduced by the defense. **Our results show that RLS misdirects the attacker in 20% to 30% of samples at each attack iteration, whereas RND misdirects only a few samples (typically one or two, with a maximum of five samples)**. This highlights the superior ability of RLS to disrupt attack algorithms compared to RND.
Rebuttal Revision Summary
We have submitted our revised rebuttal revision and would like to express our gratitude to the reviewers for their valuable comments and feedback, which have significantly contributed to enhancing our submission. In the revised version, all major changes and additions are highlighted in red. Below, we outline the changes made to our submission. - **The AAA Defense**: - We have added a new section (Section 4) to **introduce our adaptive attack against AAA-sine, a state-of-the-art defense against black-box score-based attacks that previous works, including RFD [1], have consistently failed to outperform**. Consequently, we have updated the Introduction, Abstract, and Conclusion sections to reflect this new contribution. The experimental results of our adaptive attack are detailed in Section 6.2. All changes are highlighted in red. - We have included a discussion of AAA to provide a more in-depth comparison between AAA and RLS (see Section 2.2). - Additionally, we have included the ECE comparison of our defense with AAA and other defenses (see Section 6.3 and Table 6). - We have also added a robustness comparison between AAA and RLS in Appendix F.4, demonstrating that RLS outperforms AAA. - **Additional Experiments**: - We have provided extensive experiments and evaluations in **$\ell_0$ space, using the SOTA black-box sparse attack, BruSLeAttack [2] on both CIFAR-10 and ImageNet** and have updated our Related Work and Experiments sections to highlight this. The results are presented in Appendix F.2. We also included references to other sparse attacks for decision-based attacks, for example. All of these changes are highlighted in red. - Additional experiments in $\ell_2$ space on both ImageNet and CIFAR-10 have been added in Appendix F.1. - Further experiments for $\epsilon = 0.1$ in $\ell_\infty$ space are included in Appendix F.3. - **Improvement to Experimental Setup**: - We edited our original submission to provide a more detailed description of our experimental setup (see Section 6.1). - Additionally, we have expanded Appendix A to discuss the attack and defense configurations in greater detail. - Minor changes have been made to the table captions to clarify the setup and configuration. - **Reorganization of Sections**: - To comply with the page limit guidelines following the highlighted additions, we have moved Sections 4.3 (The Method Formulation) and 5.2 (Defense Components) from our original submission to Appendices B and D, respectively. - **Code**: - We have added a more comprehensive README file to our code, along with the implementation of AAA. [1] Nguyen Hung-Quang, Yingjie Lao, Tung Pham, Kok-Seng Wong, and Khoa D. Doan. Understanding the robustness of randomized feature defense against query-based adversarial attacks. ICLR, 2024 [2] Quoc Viet Vo, Ehsan Abbasnejad, and Damith C. Ranasinghe. Brusleattack: Query-efficient score-based sparse adversarial attack. ICLR, 2024.
Dear Reviewers, We thank you for your valuable comments and feedback which helped us to improve our work substantially. We would like to take full advantage of the extended discussion period to address any remaining concerns you might have and look forward to further discussions. We also **encourage all reviewers to re-evaluate our work considering the revised version**, which incorporates **two key changes**. First, we introduce **a new contribution: the Pendulum attack, our adaptive attack against AAA**—the state-of-the-art defense against score-based attacks that prior works have not been able to surpass. Our adaptive attack **reveals a critical weakness in AAA**. Second, we have included **a more comprehensive set of experiments** to strengthen our analysis. These additions include evaluations in $\ell_0$ and $\ell_2$ spaces, larger perturbations in $\ell_\infty$, robustness comparison of AAA and RLS, the performance of our adaptive attack against AAA, and the expected calibration error of defenses. We greatly value your input and thank you once again for your thoughtful feedback. Best regards, The authors
Meta-review
This paper proposes Random Logit Scaling (RLS), a randomization-based defense against black-box score-based adversarial example attacks. RLS is a plug-and-play postprocessing defense that applies to any existing ML model. The reviewers acknowledge that RLS is lightweight, easy to use, and does not sacrifice clean accuracy. However, reviewers think the work lacks theoretical analysis, and some of the design choices seem arbitrary. The defense only works for score-based black-box attacks and is not applicable against decision-based and transfer-based black-box attacks or white-box attacks. Given the limitations of the proposed method, I recommend reject.
Additional comments on reviewer discussion
The authors and reviewers were engaged in the discussion. The authors provided substantial responses and new results, including experiments on ImageNet and a new adaptive attack against existing defenses. However, the authors did not fully address all the concerns raised by the reviewers, including a lack of more rigorous theoretical analysis, and thus, the majority of reviewers maintained scores lower than the acceptance threshold.