Adversarial Examples Are Not Real Features

Summary

This paper builds upon the work of [12] "Adversarial examples are not bugs, they are features" written by Ilyas et al. The authors of [12] introduced the concept of robust and non-robust features. According to [12], non-robust features alone can be useful for the classification task. The authors of the current paper show that such features however are not extendable to different tasks. They show this through linear probing and auto encoders. Furthermore, according to the authors of [12], robust features alone are able to create a robust dataset. However, the findings of this paper contradicts this notion.

Strengths

1. This paper extends the definition of what means for a feature to be robust vs non-robust. Instead of relying on a single classification task, they extend the definition to a task wise perspective. This allows them to define "Absolute Useful" and "Relative Useful" features. Intuitively relative usefulness represents the importance of a feature more than absolute usefulness. Using these definitions, the authors then define $\rho$-useful and $\gamma$-useful features. 2. The authors then ask a series of compelling questions. They try to find the answer to these questions empirically. In the first experiment they find that using the task definition, non-robust features perform poorly (even though not non-negligible). This indicates that non-robust features are actually not useful. Next they find that the essence of non-robust features are task-specific and they should not be considered as real features. The next question is the most interesting one. The authors find that robust features alone are not enough to for robust training. This directly contradicts with the finding in [12]. Finally, they show that different tasks capture different non-robust features. These findings together shed more light on the nature of adversarial examples and the features related to them.

Weaknesses

1. Even though the paper is pretty well written, there are some weakness in the presentation. These include several typos e.g., in (13) and (14) I believe the function $U(g, D, T)$ is mentioned incorrectly instead of $R(g, D, T)$. Similarly, in line 104 Figure 3 is mentioned, but I think that the authors meant Figure 2. Adding to these, I could not find a mention of Figure 1 in the text. This means that the evaluation framework is not properly explained. Some similar other details are also missing reducing the readability of the manuscript.

Questions

1. In Figure 2, the clean dataset always performs better than the rest. Why is this the case always? 2. What are the authors intuition behind non-negligible features having non-negligible performance in Figure 2? Does this mean that some of this features can be transferred across some specific tasks?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

1. As mentioned above the presentation of this paper can be improved. In my opinion doing so would elevate the paper further. 2. The authors ask four questions in the manuscript and answer them through experiments. It would be useful to mention these questions in the intro and comment on the usefulness of each of them.

Summary

This work analyzes the robustness of deep networks under various tasks, analyzes how adversarial attacks between models trained on different tasks transfers between them, and does so through a newly proposed framework.

Strengths

This paper delves into the robustness of models trained under different tasks (SSL training) and runs thorough analysis on CIFAR10 datasets to compare these models. As SSL trained models continue to be used, this paper's research direction is interesting and useful. The newly proposed framework seems reasonable to compare robustness.

Weaknesses

While I like the general idea of this paper, the main thrust of it was quite difficult to follow. While I am familiar with the work "Adversarial Examples Are Not Bugs, They Are Features" (or [12]), it wasn't until nearly finishing my first read-through that the submitted manuscript is heavily based on it's work. It also wasn't clear that "tasks" in this paper do not represent different datasets (with different objectives) similar to multi-task learning, but "tasks" refers to self-supervised learning. These previous points could be made much clearer. Furthermore, it appears that the paper may have been written under time constraints, as there are multiple instances of typographical errors throughout the text ("datatset" and "hekp" for example). Taking the time to carefully proofread the content would greatly enhance its clarity and overall presentation (e.g., Fig 3 referenced on page 4, but shown on page 6). Lastly, many technical details are left out of the paper which would seriously help understanding: a brief overview of each of the SSL methods, how are the datasets constructed, and where linear probing is performed. The proposed "theory" framework appears to be heavily based off [12], and it is more of a list of definitions rather than offering any theoretical insights. While these definitions are useful, theoretical insight could be given by analyzing how different (SSL / non SSL) tasks could effect the robustness more comprehensively. Some of the large claims in this paper, that the robust datasets are in fact not robust, are not quite backed up by the experiments of the paper. The robust dataset was generated by using a classifier, to then train a classifier. I'm not sure why it's surprising that using that robust classifier specific dataset doesn't give robustness to SSL trained models. Here's a natural experiment: 1. Train an SSL model on the clean dataset 2. Fit a linear probe for classification 3. Use that linear probe to generate the robust dataset following [12] 4. Train a new SSL model on that new robust dataset 5. Attack that model I would be very curious to see what Table 3 would look like after following the above steps. Finally, results on using CIFAR-10 have questionable generalization. It is a little surprising that the "Restricted ImageNet" from [12] was not used for experiments.

Questions

See Weaknesses for items to address.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

1 poor

Contribution

2 fair

Limitations

See above. No ethical statement, a little could be said about dataset robustness and safety.

Summary

This work challenges the theory on "robust" and "non-robust" features from Ilyas et al. With an extended and more generalized formulation, the authors show that "non-robust" features are indeed very task-specific, and that even supposedly "robust" features are mostly task-specific and hardly provide robustness. The authors experiment with various self-supervised models on CIFAR10 (and its robust/non-robust variants) and answer multiple interesting research questions.

Strengths

- The original work from Ilyas et al. did mention that the "robust" features are task specific, but this work explicitly tests and shows that "robust" features are not necessarily truly robust, and and only really "task-robust". In some sense, there are 2 levels of generalization- this work shows that robustness at the second stage (same X, Y distribution) does not imply robustness in the first stage (same X, different Y distribution). - I like Eq (9) - providing a formulation that is relative to the choice of $g$. - The conclusion in L199-200 is fitting, and ideally should have been a stress point in the original paper on robust features. The authors here have done a good job of making this point explicit to clear up the misconception around "robust" features. The fact that the original paper did claim robust features to be a property of the dataset is in direct contradiction to results from this work.

Weaknesses

- My biggest concern is the interpretation of "robust" and "non-robust" features. The authors challenge the claim made by Ilyas et al. about the actual robustness of features. However, the cited paper does not claim that the "robust" features they identify are universally robust, only that they are robust for the classification task at hand. This is not surprising- 'eye color' would be a robust feature for person identification, but not for smile detection. These features are statistical patterns that ere useful for the **given** task but not of any use otherwise, so not surprising that they do not generalize to multiple tasks. Even on L112, the authors claim "we believe that the existence of non-robust features is task-reliant", which is what Ilyas et al. also say. - L144: "...meaning that the adversarial perturbations are almost meaningless to DDPM". This can also mean that the attack is not potent enough. Also, there is no reason to believe that adversarial perturbations should transfer across tasks. The objective when adding perturbations for one task is unrelated to the wanted objective for another task- perturbations meant to fool smiling/non-smiling have no reason to influence classification scores for straight/curly hair, for instance. The former would most likely perturb areas around the face, while the latter would likely look at features close to the head. - All experiments are focused on CIFAR-10 and its robust/non-robust variants. I would like to see at least one more dataset to be more confident in the generalization of claims made in the paper. ## Minor comments - This paper assumes that the readers are familiar with the work on "robust features" referenced throughout the paper. Please give a brief summary of the referenced paper (from MadryLab) - their crux and notion of "robust"/"non-robust" features in the Introduction itself. - L19: "...and it becomes natural for papers to use terms like..." - please give some examples of papers that indeed do this. - L35: "experimente" -> "experiment" - L41: Please provide a list of contributions. Although posing these research questions is a good way to pique interest, they should not not be left as unanswered questions until the end of the paper. - Eq (12) seems to be missing the term $T$ - Figure 2 missing axis labels - L164: "...exceeds 80%" - what is the ASR of a robust model for the same attack? - L169-172: "First, the attack method.......from gradient obfuscation" - this might mean that the "robust" features are not entirely robust and have some noisy non-robust features in them too. - Figure 4: please provide a heatmap legend. This is an interesting figure- why is cosine similarity of perturbations a good metric? Why not look at transferability rates instead. The latter would be a better and more direct indicator. - References formatting is mixed: conference names are mixed in lower-case, capitalized, etc. For instance, "Are adversarial examples inevitable" appears in ICLR but the arxiv version is cited. Some conference names are in full while for others abbreviations are used.

Questions

In Eq(11) the same $\epsilon_0$ seems to be added to added for all tasks? This does not seem optimal at all, since different tasks may require different levels of noise. For instance, a binary classifier would require much more noise (happy/sad classifier) as opposed to a more fine-grained task (person identification).

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

2 fair

Limitations

N/A

Summary

In this study, the authors dispute the argument from "Adversarial examples are not bugs, they are features," where it was suggested that adversarial examples exist due to non-robust yet useful image features. They used self-supervised learning algorithms on both robust and non-robust datasets to test this theory. The findings contradicted the hypothesis, revealing that the self-supervised models trained on non-robust datasets didn't generalize well, indicating that non-robust features aren't universally useful for different training setups. Additionally, it was demonstrated that non-robust features, while beneficial for classification, aren't helpful for reconstruction, highlighting their task-specific utility. Moreover, models trained solely on robust features lacked robustness, showing high vulnerability to AutoAttack. This contradicts the original study's findings, as these models failed to exhibit robustness even when trained exclusively on robust data. Lastly, an examination of cosine similarity between the adversarial attack directions of various self-supervised models showed significant differences, suggesting adversarial attacks aren't easily transferable between different training setups. This indicates that non-robust features are generally not very useful and are model-specific rather than dataset-specific.

Strengths

- This paper principally examined the arguments made by [1], by training models in a self supervised manner on non-robust and robust datasets in [1]. - I think the finding the models trained on robust features alone are not robust is already a very important finding. As [1] is a paper that the community has been very interested in, it is important that other papers try to reproduce and validate the results. [1] Ilyas, Andrew, et al. "Adversarial examples are not bugs, they are features." Advances in neural information processing systems 32 (2019).

Weaknesses

- I think the authors should really expand on 6.2 and table 3. The results is truly surprising and strongly contradict the original finding in [1]. For example, the authors suggest that the use of AutoAttack is the main cause for the discrepancies with the finding of the original paper. The author should verify whether the current model is vulnerable under PGD and CW attack. If the current model is also vulnerable under PGD and CW attack with more steps, which I think the current model is going to be. Then what are the conditions for reproducing the original finding, and how does the author's setting differ from it?

Questions

Could the author shows whether the current model in Table 3 is vulnerable to both PGD and CW attack? Also, could the authors ablate the experimental differences between the original paper and the current on?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors have addressed the limitations adequately

Concerns addressed

The authors have addressed my concerns quite well- I will update my rating to reflect this as soon as the option becomes available. I have no further questions for the authors :) Good luck!

Response to author rebuttal

I thank the authors for taking the time to address the reviewers' concerns. In particular the addition of the TinyImageNet-200 results was very helpful. I would encourage the authors to make those results a part of the main manuscript. After reading the other reviews and the rebuttals I have decided to update my rating.

Thanks authors

I'm mostly convinced by your rebuttal. I am still not a fan of the word "task" to describe different SSL losses, and I do worry about the general readability of the paper. But those are not content issues. I'll raise my score.