Spatial-frequency channels, shape bias, and adversarial robustness

Summary

This study applies a well-established procedure from psychophysics, critical band masking, to an object recognition task on which human and a wide range of neural networks are compared. The core finding is that while humans use roughly the same spatial frequency channels for object recognition as they do for e.g. letter / grating recognition, all 76 investigated neural networks use much wider channels (more than twice as wide as the human channel). This means that humans and neural networks rely on different spatial frequencies for object recognition.

Strengths

* Overall, I really liked this study - it has a simple yet effective protocol, clear findings, and approaches a novel question through sound experiments. * The proposed approach is applicable to the analysis of future networks, i.e. it is sufficiently general such that it can be used by follow-up studies / on general architectures * Successful transfer of psychophysical method to the study of neural networks * The paper is very well-written, easy to follow, clearly structured and has good visualizations * Human data will be made publicly available upon publication

Weaknesses

Significance testing: * Figure 6 does not correct for multiple comparisons. I would like to know which of the plotted effects still hold after Bonferroni correction by dividing the alpha level (.05) by 18, i.e. the number of comparisons (6 plots x 3 fits), resulting in a corrected alpha level of 0.0027777. I would encourage the authors to update the figure with a corrected version. Human experimental weaknesses: * In human visual perception, spatial frequency is measured in cycles per degree of visual angle and depends on the distance to the monitor. If I understand correctly, the setup is employed with unknown distance between observers and monitor. * No information on monitor calibration, which is very important in experiments where stimuli of reduced contrast are shown. E.g., many monitors have settings that "enhance" contrast, which is not unlikely to have an impact on human experimental performance. Did the authors take this into account? * Human experiments with reduced contrast are best performed in a setting where contrast adaptation can take place. In the present study, however, the screen constantly changes from grey to bright white and back to grey again. * Given the online study setup, how did the authors ensure normal / corrected-to-normal vision (beyond self-reporting, where online participants have monetary incentives for inaccurate reporting)? * Two of the 16 observers (supplementary Figure 2) do not seem to perform well at all on the simple task

Questions

### Questions: * What is going on with the two outlier observers in Figure 2 (1st column, bottom two rows)? * Given that some networks have severely reduced performance at 20% contrast, I'm wondering whether there might be a possible interaction of the experimental findings with the contrast level. As a supplementary experiment, it might be interesting to test the core findings separately for (a) networks which only show a minor performance detriment when reducing images to 20% contrast and (b) the rest of the networks. ### Major opportunities for improvement: (The study is already quite solid, and I am in favor of publication. These suggestions go beyond what is currently done in the paper. If at least one of them can be addressed, I would be happy to increase my score further.) * "we propose that critical band masking should be added to the toolbox of metrics for model-human comparison.": I agree, and I was wondering whether the authors plan to facilitate this. For instance, they could release their stimuli as part of an open-source testbed (e.g., as a fork of / addition to the model-vs-human toolbox, or in any other way). * The authors show that properties of critical bands are correlated with important properties like shape bias / adversarial robustness. Going beyond a correlation, I was wondering whether a causal link could be established, e.g. by reducing the signal-to-noise ratio of non-human channels in the input during training, leading to changed critical bands (e.g., more human-like ones), and then showing that the properties of interest (shape bias / adversarial robustness / ...) change accordingly. * The reasons for the human-model divergence are not yet understood. It would be interesting to know why this divergence arises, and whether it is a property of the architecture, the training procedure, or something else. ### Minor questions: * line 86: should this be "human object recognition"? * why "roughly equally distributed across all 16 categories"? What was the exact distribution, and why was it not exactly equally distributed? (Background: performance can be above chance level 1/16 when categories aren't exactly equally distributed)

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

2 fair

Presentation

4 excellent

Contribution

4 excellent

Limitations

The authors don't have a section on limitations. In accordance with the checklist, the authors are encouraged to describe the limitations of their approach, in particular related to the experimental choices mentioned in the "weaknesses" section above.

Summary

The authors propose to measure the differences in how humans and models rely on spectral information to recognize objects by using critical band masking to ablate. They conduct a human study revealing that humans detect objects with the same channels they use to detect letters and grating. On the other hand, models show a significantly wider channel. They further correlate the properties of the channel to robustness and shape bias, e.g. showing that AT results in an even wider channel.

Strengths

### Originality The paper explores a less common yet highly important research direction. Some "model" findings were partially reported in literature, but for the most part the findings are highly original. ### Quality The submission is of high quality (minor weaknesses see below). The authors rigorously conduct (well-documented) human experiments and benchmark 76 ImageNet models to arrive at their conclusions. ### Clarity The paper is very well written. Key points are communicated clearly through writing and figures. ### Significance The authors share many findings, that may be of great interest to the NeurIPS community. In particular, they measure the human critical channel and show that models measure higher bandwidths. Adversarial Training increases the channel even further. Generally, they find correlations between channel properties and shape-bias/robustness that may lead to a new perspective on robustness but also shows another misalignment between humans and models.

Weaknesses

### Method - Evaluation only on ImageNet: I never thought I would make this as a weakness but here we go ... previous work has shown that the frequency band of adversarial attacks is not always HF (the authors arrive at the same conclusion in L230ff) but also highly depends on the dataset [2,3,4]. Given that this paper "only" presents an empirical evaluation I am not sure if the findings (about models and in particular robustness) scale to other datasets. At the very least this is a limitation that should be discussed. - Poor choice of robust networks: The authors choose $\ell_2$-AT trained ResNets as robust networks and attack them with $\ell_\infty$ attacks. However, AT is notoriously poor at generalizing to new attacks. A better choice would have been $\ell_\infty$-AT trained ResNets. There may also be some differences in the evaluation as [1] showed that the norm differently affects performance on high/lowpass data. I don't expect this to flip observations, but it would be a more appropriate choice. - Minor: uncommon choice of parameters for robustness analysis. A more common choice would be $\epsilon=4/255$ via 20-step PGD or better the AutoAttack suite. This probably won't impact the observations at all but may be easier to relate to. ### Novelty - Some claims/observations in the submitted draft were already published. This does not invalidate the (exceeding) contributions of this paper, but the respective works should at least be mentioned. [2] already performed band-masking to study the critical bands of CNNs, [1] discussed how frequencies interact with shape-bias. ### Others - The authors use 76 pretrained models only partially citing them. The Paper Checklist clearly states to cite all the used assets. - Authors do not discuss limitations - Minor: The authors repeatedly claim that transformers are shape-biased. This is not inherently true. It may hold if pretrained on large datasets such as ImageNet21k, but out of the box they don't perform much better, e.g. [1] shows that XCiT just barely performs better than ResNets in shape-bias when only trained on ImageNet1k. [1] Paul Gavrikov, Janis Keuper, Margret Keuper. "An Extended Study of Human-Like Behavior Under Adversarial Training". CVPR-W, 2023. [2] Antonio A. Abello, Roberto Hirata, and Zhangyang Wang. "Dissecting the high-frequency bias in convolutional neural networks". CVPR-W, 2021. [3] Remi Bernhard, Pierre-Alain Moellic, Martial Mermillod, Yannick Bourrier, Romain Cohendet, Miguel Solinas, and Marina Reyboz. "Impact of spatial frequency based constraints on adversarial robustness". IJCNN, 2021. [4] Guillermo Ortiz-Jimenez, Apostolos Modas, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. "Hold me tight! influence of discriminative features on deep network boundaries". NeurIPS, 2020.

Questions

- Are there any insights on whether improving the alignment between the model critical channel compared to humans would improve robustness or shape-bias, i.e. is there evidence of causality and not only correlation? - Did the authors attempt to encourage alignment in training? - Are there any observations on other datasets?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

3 good

Contribution

4 excellent

Limitations

The authors do not discuss limitations explicitly. One salient limitation is the evaluation of ImageNet. As discussed above certain frequency properties may not carry over to other datasets. I don't see any potential negative societal impact of their work.

Summary

This paper presents an analysis of neural network response to object classification from the perspective of frequency components. The authors use an analytic tool based on critical band masking, which is a technique borrowed from neuroscience. The focus of the network to use certain frequency bands for object classification is related to the frequency capabilities of the human visual system, then with shape bias and adversarial robustness of classification networks. The contributions/novelty is in the neuroscientific technique used to analyze the network responses and behavior. The contributions are related to comparison of the frequency bad usage in the human visual system (HSV), neural networks and adversarially-trained networks, with limited contribution in terms of practical insights or indications/suggestions on how these observations would be usable.

Strengths

- Use of a novel approach to analyze neural networks: the critical band masking technique is borrowed from neuroscience (very popular in auditory system analysis) and used to analyze the response of networks. The approach is novel and interesting. - Extent of models considered in the analysis (although the transformers are only slightly covered)

Weaknesses

- Usefulness and outlook of the results: while the experimental results present an interesting view on the neural networks frequency selectivity for object classification, their significance and possible usage are not clear nor explicitly covered. Networks have a different behavior/frequency selectivity (not surprisingly I would say) nut no indication of how one should take over these findings and use them. - Relation between HVS and neural networks: the paper reads with an implicit assumption that the human visual system and neural networks for vision tasks are comparable systems. They are actually very different (with the second showing only some conceptual resemblance with the first), in terms of processing the data, energy-efficiency, ‘training’ etc. The results show that they behave differently (as expected) but do not address substantial differences between the two systems. I see a missed answer to important questions, as ‘why should we compare these systems and what do we gain from this analysis’? - Details about the methodology are missing: e.g. is the analysis done taking into account both magnitude and phase of the spectrum, or only the magnitude? Phase is for instance shown important for object classification [1]. - Placement in the literature is very limited – references are outdated: missing references of works that study the network from a frequency perspective, relating the frequency spectrum with robustness and bias problems ([2,3,4,5], to mention a few). [1] Chen et al. Amplitude-phase recombination: Rethinking robustness of convolutional neural networks in frequency domain, ICCV 2021. [2] Yin D. et al. A fourier perspective on model robustness in computer vision. In NeurIPS, 2019. [3] Maiya et al. A frequency perspective of adversarial robustness, 2022. [4] Wang et al. Frequency shortcut learning in neural networks. NeurIPS 2022 Workshop on Distribution Shifts [5] Wang et al. High-frequency component helps explain the generalization of convolutional neural networks. In CVPR, 2020.

Questions

- Can the authors clarify details about the method and experimental analysis, such as the use of magnitude and phase of the spectrum? - Can the authors clarify the motivations why neural networks based vision models have to be compared with the human visual system of the brain, and why aspects such as energy efficiency, memory, and ‘training’? are not taken into account? - How does the work related to actual existing work on frequency analysis of neural networks in vision tasks? - How the achieved results fall within the current progress in understanding neural network learning and application, and how they can be used to improve models for vision task? - Are 16 human observers enough to have significant results? This is glossed over very briefly in the paper, but not analyses are provided to support the claims.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The authors do not discuss limitations explicitly. About ethics, the authors say that humans are used for experiments and that a consent form has been signed, but there is no evidence of how this form is written, what the users agree with, and who approved the experiments.

Summary

This work presents an empirical analysis of the sensitivity of humans and neural networks to critical band masking. This is the process of perturbing specific spatial frequencies, and determining which frequencies and at what magnitude, will lead to recognition errors. This in turns identifies the frequencies/magnitudes that a visual system “filters out” or “relies on” when performing recognition. The main finding is that neural networks are sensitive to a larger set of noise frequencies/magnitudes than human participants. Further, the paper presents evidence that adversarial-trained networks are sensitive to an even bigger set of frequencies/magnitudes, and correlation-based analyses indicating that more human-like networks that are sensitive to a narrow set of frequencies (smaller bandwidth) have a bigger shape bias.

Strengths

Originality - Critical band masking-based analysis has previously been done to analyze recognition of letters and gratings. The novelty for this paper comes from its critical band masking-based for ImageNet-like object recognition for 16 humans and 76 different neural networks. Quality - The design of the study is mostly sound: 1100 images from 16 categories of ImageNet with various kinds of added noise are used as the test set. Careful steps were taken to avoid clipping as a result of added noise. 16 participants are sufficient due to the small variability between subjects (Figure 2 in supplement). - The evidence for the claims of bigger channel bandwidth than humans, inverse correlation of bandwidth and shape bias, and increased bandwidth with, are sound regarding the 76 networks evaluated in this paper. Figures 4 5 6 contain clear evidence of these claims Clarity - The paper is well written and mostly easy to follow. The plots in Figure 5 and 6 are quite condensed, containing a lot of information, but with the text and captions the reader can mostly parse them well.

Weaknesses

Quality - The practical motivation behind using grayscale images reduced to 20% contrast makes sense, however, this represents a testing condition outside of the training data domain of these neural networks. A control experiment where the performance discrepancy between testing on standard RGB data and contrast reduced grayscale is quantified would justify this choice of test setting. - While the paper does investigate a diverse set of networks, one confounding factor that at least should be mentioned and ideally should be investigated in an analysis paper like this is the effect of training data and data augmentation on spatial frequency sensitivity. The design of the neural network itself is not the only factor. * The neural networks used in this paper are primarily trained on ImageNet, but there are other datasets, for example Office-Home, with images from different domains for each object, and therefore different spatial frequency characteristics. Training on such data will likely result in different spatial frequency sensitivity. * There might be straightforward steps like band-pass filtering or adding noise to the input data at training time, that might effectively reduce the range of frequencies a model is sensitive to. - The paper lacks a related work section. While it does mention a variety of prior work throughout the draft, it would be beneficial to include a separate section that would cover prior work on topics like: spatial frequency sensitivity of humans and neural networks, shape bias, adversarial robustness, and comparing humans and neural networks more broadly. The main benefit is to help the reader understand exactly where the proposed work lies relative to prior methods. Significance - While it is interesting to see a comparison of spatial frequency sensitivity of networks to humans for object recognition, and to identify it as an emerging phenomenon for a variety of ImageNet-trained models, I’m concerned about what exactly is the significance of these findings for deep learning practitioners. How should researchers rethink their work going forward as a result of these findings? - As mentioned in the quality section, after reading this paper there is a sense of it not going quite far enough in the analysis. How pervasive is the phenomenon of a wider spatial frequency sensitivity of neural networks? It would be more impactful if it is demonstrated that this is a general phenomenon despite training data or simple mitigation strategies. - The observations about adversarial robustness are focused on one type of white-box adversarial attack, which is the least practically plausible.

Questions

- What exactly is the significance of the findings presented in this work? What should be the main takeaway for researchers working on deep learning methodology for visual recognition, regarding how their future work should be affected by these findings? Is it necessarily better if the "channel" (as defined in this paper) is more human-like, and if so, why? - Given that filtered noise perturbation significantly alters the visual appearance of an image, so much so that it appears to be quite easy to identify images with such an attack, what is the significance of identifying the susceptibility to such attacks of adversarial-trained models (described in L298)? - What is the significance of the current findings abound frequency sensitivity, in light of the fact that the effect of training dataset choice and some basic techniques could be used to improve it?. How pervasive is this emerging phenomenon of a wider spatial frequency sensitivity of neural networks?

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The authors have adequately addressed the limitations.

Summary

I have read the authors' rebuttal and will maintain my already very high rating. This paper discovers a new fact (which should be replicated by others) about human vision: That we use the same frequency band for objects as we do for words and gratings. It seems remarkable that this has not been tested before, but a cursory google search didn’t turn up any previous result like this. Second, the paper shows that 76 deep network vision systems use a much wider band of frequencies than humans. Finally, the paper shows that aspects of these frequencies predict both shape bias and robustness against adversarial examples.

Strengths

-This is one of those papers that one wishes one had written. The result about the human bandwidth is *Nature*-worthy! -The sequence of experiments: First human psychophysics, then network “psychophysics”, then showing that these measurements of noise-sensitivity (mean, std dev., and peak noise sensitivity) account for fairly large amounts of variance in the networks’ adversarial robustness and shape-sensitivity are both logical and surprisingly revealing. -The fact that these measurements account for 53% of the variance in shape-bias is remarkable, given that these are relatively low-level, and don’t have any necessary relationship to shape.

Weaknesses

Just a few comments here: Include the heat map key in the supplementary material (figure 2). You should run some more experiments with adversarially-trained networks (see below).

Questions

The x axis for peak noise sensitivity in Figure 6 seems incorrect, based on the definition in Figure 2F, For example, for humans, it looks to be 1/(0.03), or about 33. It’s only after checking the supplementary material that this is made clear. It’s actually 1/(0.02*2^A), where A is the fitted parameter. This should be made clearer in the main text. Did you use MTurk’s ratings of workers? It seems like subjects 9 and 13 were only marginally engaged in the task. I would have thrown them out. I think you should reverse Figures 1 and 2. Also (if possible) 5 & 6. In both cases, you refer to the later one before the other.

Rating

10: Award quality: Technically flawless paper with groundbreaking impact, with exceptionally strong evaluation, reproducibility, and resources, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

Despite using 76 different networks, they only use ResNet50 for the adversarially-trained networks. It would be good to add a couple of other networks of different types (e.g., ViT, etc.) to see that this still holds. This could be added to the supplementary material.

Thanks

I would like to thank the authors for their detailed reply. I particularly appreciated that they corrected the alpha level for multiple comparisons and provided the updated plot, and explanations regarding experimental methods. Overall, I am satisfied with the author's response and promised edits to the paper. Some experimental weaknesses remain - e.g. in terms of monitor calibration hoping is great, measuring is better - but those are inherent to the online study design and cannot be expected to be resolved in a rebuttal. My final score is "7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations." and I strongly support publication at NeurIPS.

Thank you for your efforts!

Thank you for your efforts! The rebuttal addresses most of my issues. Remaining issues: 1. I am looking forward to the updated robustness results 2. Regarding "Evaluation only on ImageNet". I am afraid that just updating the conclusion is not enough. There are multiple sections in the paper that strongly suggest that your findings scale to the detection of objects in general (e.g., L38). It would be more honest to clarify the scope of your results early on.

Thanks for the answers.

Dear authors, I sincerely thank you for your clear answers, especially regarding putting the significance of your work into perspective. Your rebuttal is very clear and with good argumentations. I also read the reviews made by colleagues, and your rebuttal to them. I liked very much the perspective you gave about the relation between neural networks and human visual system, and how your results might be useful to improve networks in the future. About the ethics statement: I realize that my request made no sense wrt confidentiality of the submission. I will remove the flag. In summary, I appreciate the clarifications and I am willing to raise my score to Accept.

Typo in new related work section

ImageNet-pretrained convolutional networks, on the other hand, are biased towards shape (Geirhos et al., 2018, Baker et al., 2018). -> ImageNet-pretrained convolutional networks, on the other hand, are biased towards texture (Geirhos et al., 2018, Baker et al., 2018).

Robustness?

Any there any updates on the reevaluated robustness?