Towards Reliable Evaluation and Fast Training of Robust Semantic Segmentation Models

Reply to Reviewer CxEd (part 1)

We thank the Reviewer for the positive and detailed feedback. In the following we address the weaknesses and questions raised in the review. &nbsp; **Q: The advantages of the proposed three losses are not well depicted. It would be better if the author could discuss under different scenarios which proposed loss is the best for attacking. Some visual examples would be better.** The (theoretical) discussion of benefits and weaknesses for each loss in Sec. 2.2 suggests that one main difference among losses is how they balance the weight of different pixels in the objective function. On one extreme, the plain cross-entropy maximizes the loss for all pixels independently of whether they are misclassified, and assign them the same importance. Conversely, the masked losses exclude (via the mask) the misclassified pixels from the objective function, with the danger of reverting back the successful perturbations. As middle ground, losses like the JS divergence assign a weight to each pixel based on how “confidently” they are misclassified. We conjecture that for radii where robustness is low, masked losses help focusing on the remaining pixels, and already misclassified pixels are hardly reverted since they are far from the decision boundary. Conversely, at smaller radii achieving confident misclassification is harder (since the perturbations are smaller), and most pixels are still correctly classified or misclassified but close to the decision boundary: then it becomes more important to balance all of them in the loss, hence losses like JS divergence are more effective. This hypothesis is in line with the empirical results in Table 2. We are happy to add this more detailed discussion to the paper. Finally, if the Reviewer could clarify what they mean for “visual examples”, we will address that point directly. &nbsp; **Q: The organization of this paper is confusing. The introduction of some existing works like APGD should be put in the related works. An overview of the method's structure could be added to the beginning of section 2. As section 2 mentions PIR-AT s many times, a short description about PIR-AT model is also necessary.**\ **Q: As many related works and ablation studies are mixed in the method part, it is difficult to distinguish the contributions of this work from previous works.** Thanks for the suggestions about the presentation. We agree with the Reviewer that the definition of PIR-AT comes late in the paper, we will add a short description earlier on in the revision. We will also add more detail of SEA at the start of Sec. 2, and discuss APGD in the related works. &nbsp; **Q: The SEA is not presented clearly. Why four losses perform worse than all six losses is not analyzed. How the current four losses are selected is not mentioned. And SEA doesn't discuss how to balance different losses.** We introduce the details of SEA in Sec. 2.4, combining the loss functions analyzed in Sec. 2.2 and the optimization algorithm from Sec. 2.3. SEA is based on a worst-case evaluation: this means that, for each image, the strongest attack among those obtained running APGD on the different losses is considered. Therefore, having more losses can only improve its performance (since it has a strictly larger set of attacks to choose among). However, each additional loss requires additional computational cost. Then, as mentioned in Sec. 2.4, we select 4 losses which allow our ensemble SEA to perform already on par with using all 6 losses, while saving ⅓ of runtime. This is further illustrated in one of the ablation studies in App. C.2, which shows that the improvement given by 6 over 4 losses is minimal (<0.1% in robust accuracy). Finally, since the losses are used for independent runs, it is not necessary to balance them. &nbsp; **Previous works[1] have proven that using a better robust initialization model could improve the task model's robustness. How PIR-AT is different from the existing practices is not well presented.** [1] studies the effect of self-supervised pre-training for robustness of image classifiers: they show that adversarial pre-training has no significant advantage over clean pre-training to provide best robust accuracy on CIFAR-10 when full adversarial fine-tuning is used (i.e. all network parameters are updated with adversarial training), as illustrated by Table 3 in [1]. In our case, we show that leveraging pre-trained image classifiers which are publicly available, e.g. in RobustBench, we can largely improve the performance of adversarial training on semantic segmentation tasks. This means that in this scenario, unlike in [1], using a robust backbone has a significant impact on the resulting robustness in the target task.

Reply to Reviewer tMpk

We thank the Reviewer for the positive feedback. In the following we address the weaknesses pointed out in the review. &nbsp; **Q: This paper needs to discuss more about PIR-AT. It is only mentioned that this paper proposes Pre-trained ImageNet Robust Models. What is this method in detail?** With PIR-AT (as described in Sec. 3.1) we propose to use a robust image classifier, i.e. obtained with adversarial training, on ImageNet as initialization of the backbone of the segmentation model. Such initialization allows adversarial training on the segmentation task to achieve higher robustness than existing methods, even with lower computational cost. As shown in Table 3 and Table 4, our PIR-AT models outperform DDC-AT (Xu et al., 2021), SegPGD-AT (Gu et al., 2022), and match or improve over AT from clean initialization with 4-6x fewer training epochs. &nbsp; **Q: What is the motivation to limit this paper to focus on l_{\limit} threat model?** We focus on the $\ell_\infty$-threat model since this is the most popular in the literature: in fact, for image classification, the large majority of defenses reported in [RobustBench](https://robustbench.github.io/#div_imagenet_Linf_heading) are for $\ell_\infty$, and all of those on ImageNet (which we use as initialization of the model backbone in PIR-AT). Similarly, the closest prior work for robust semantic segmentation, SegPGD (Gu et al., 2022), focuses on $\ell_\infty$-bounded attacks. However, we think that SEA might be easily extended to other $\ell_p$-threat models since APGD provides versions for such cases, which could be an interesting direction to explore for future work.

Reply to Reviewer pqj9

We thank the Reviewer for the feedback. Below we address their concern about novelty raised in the review. &nbsp; **Q: My major concern is the lack of originality and novelty in the paper. All the losses, optimizations tricks, and robust models utilized in the paper are obtained from the existing literature (authors have cited the prior works sufficiently). There are no novel methodological contributions presented in the paper. Paper detail as different existing components collectively utilized to obtain better results. The findings shown in the paper are interesting. However, I believe that they alone do not support to meet the standards for accepting at a conference. Novel methodological contribution regarding losses or obtained robust models would be appreciated.** We appreciate that the Reviewer finds our results interesting and our paper well written. However, we think that the contributions of this paper are more than sufficient - in fact this paper sets completely new standards regarding robust semantic segmentation: 1. on the level of robustness evaluation, our attack ensemble SEA shows that even the previously best attack SegPGD overestimates robustness by more than 17% (Table 1, PIR-AT model, evaluation at 12/255). As already observed with AutoAttack, a good and reliable attack for robustness evaluation enables better comparisons of new defense techniques and thus faster progress in the field. 2. we show how to train robust semantic segmentation models and improve by more than 20 points in mIoU compared to the SegPGD-AT paper (their reported numbers are even for a weaker attack than ours). In the previous paper DDC-AT (Xu et al., 2021), a sophisticated method was presented, which unfortunately proved to be completely unrobust. Therefore, we believe that it is an important contribution of this work to show how to train robust semantic segmentation models using pre-trained robust ImageNet backbones, and even save computational time. Also, this shows that seemingly strong methodological contributions can end up being completely useless. 3. we make all our code and models available (in contrast to SegPGD-AT) to foster research in this area as well as reproducibility. Additionally to these points, we have clear methodological contributions as we analyze why the plain cross-entropy loss, one of the standard losses for attacks for image classification, does not work well for semantic segmentation. At the same time we show that the Jensen-Shannon divergence, a loss not used for adversarial attacks before, has exactly the properties needed for semantic segmentation (the gradient vanishes as the pixel is maximally misclassified) and thus does not require any masking. Thus while the losses themselves are not novel, there is a clear novelty in our analysis and in the justification why these losses are good for semantic segmentation.

Reply to Reviewer hjSQ (part 1)

We thank the Reviewer for the detailed feedback. In the following we address the weaknesses and questions raised in the review. &nbsp; **Q: My major concern is the limited novelty, as the explored loss functions are not new. Although JS divergence, masked CE loss, and masked spherical loss have not been commonly used in the context of segmentation attacks, in my view, this appears to be a simple 'plug and play' of loss functions** In the discussion around the usage of different losses, we think we have clear methodological contributions: in fact, we analyze why the standard cross-entropy loss, one of the standard losses for attacks for image classification, does not work well for semantic segmentation. At the same time we show that the Jensen-Shannon divergence, a loss not used for adversarial attacks before, has instead the properties needed for semantic segmentation (the gradient vanishes as the pixel is maximally misclassified) and then does not require any masking. Thus, while the losses themselves are not novel, there is a clear novelty in our analysis and in the justification why these losses are good for semantic segmentation. Moreover, we uncover the complementarity of different losses, which is key, together with the improvements in the optimization algorithm, for our ensemble attack SEA to outperform the existing SOTA attacks by more than 17% (Table 1, PIR-AT model, evaluation at 12/255). &nbsp; **Q: The conducted attacks are white-box, and the absence of black-box evaluation is a significant limitation** To extend the evaluation to black-box methods, we tried to adapt Square Attack [(Andriushchenko et al., 2020)](https://arxiv.org/abs/1912.00049), a SOTA score-based black-box attack for image classification, by using its random search-based algorithm to optimize our JS loss. However, the resulting attack has poor performance even on clean models. We hypothesize that the localized updates in Square Attack are not sufficiently effective to optimize pixelwise losses. Similarly we tested using gradient estimation via finite difference [(Ilyas et al., 2018)](https://arxiv.org/abs/1804.08598), but could not get performance close to that of white-box methods. We showed in the paper that balancing gradients of different pixels is already difficult with exact gradients, then we argue that having only approximated gradients is expected to make optimization even more challenging. This shows that finding an effective black-box attack for semantic segmentation requires designing task-specific algorithms, which could be the topic of an independent paper. As additional black-box evaluation, we tested transfer attacks from less robust models to the PIR-AT. In particular, we run APGD on the Masked-CE loss on Segmenter models obtained with either clean training or AT (5 steps) on ADE-20k. We then transfer the found perturbations to our PIR-AT (5 steps, 128 epochs), and report robust accuracy and mIoU in Table A, together with the results of the white-box SEA on the same model (from Table 3 of the paper) as baseline. We observe that the transfer attacks are far from the performance of SEA, which further supports the robustness of the PIR-AT models. We are happy to add this evaluation (potentially expanded to include more models) in the revision of the paper. **Table A:** Robust average accuracy and mIoU given by transfer attacks on ADE-20k dataset. In boldface the white-box baseline by SEA. |source model | target model | 0 | | 4/255 | | 8/255 | | 12/255 | | |:--------------|:-----------|-----------:|--------:|-------:|--------:|-------:|-------:|--------:|-------:| | | |aAcc | (mIou)| aAcc | (mIou)| aAcc | (mIou)|aAcc | (mIoU)| | Segmenter clean | Segmenter PIR-AT | 69.1 | (28.7) | 68.8 | (28.3) | 68.6 | (28.0)| 68.3 | (27.8) | | Segmenter AT | Segmenter PIR-AT | 69.1 | (28.7) | 66.3 | (26.0) | 63.1 | (23.8) | 57.4 | (19.9) | | **Segmenter PIR-AT** | **Segmenter PIR-AT** | 69.1 | (28.7) | 54.5 | (16.1) | 32.8 | (7.1) | 8.6 | (1.8) |