Emergence of Shape Bias in Convolutional Neural Networks through Activation Sparsity

Summary

This work is motivated by the observation that deep neural networks may have a bias towards texture in classification decisions. The paper proposes a top-k sparsity selection process that is shown to result in emergence of shape sensitive neurons while improving performance across a variety of tasks.

Strengths

1. The demonstration that shape encoding occurs when the proposed sparsification is applied even in the absence of training is an important result and contribution. 2. The paper demonstrates the value of this operation across a variety of tasks from recognition to synthesis and connections to neuroscience. 3. The paper is generally well presented and gives a variety of examples that present a strong case for the claims made.

Weaknesses

1. There are some editorial changes that would benefit the paper (e.g. which numbers are bold in tables, when is SD bold etc.) and some places in the text where the description could be refined. 2. It would be nice to see something like figure 4 for the ResNet variants given their importance in the literature 3. Some references might be added (e.g. Md Amirul Islam, Matthew Kowal, Patrick Esser, Sen Jia, Björn Ommer, Konstantinos G. Derpanis, Neil D. B. Bruce, Shape or Texture: Understanding Discriminative Features in CNNs, ICLR, 2021.)

Questions

1. How does the rank based selection for sparsity compare with 'softer' methods for inducing sparsity (e.g. L1 regularization)? 2. Is there any means of knowing the proper value for the k hyperparameter?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

4 excellent

Limitations

The authors are clear about limitations of the work.

Summary

The paper proposes to use activation sparsity induced by the top-k operation to address the recently identified hard problem in DNNs, the texture bias. The operation is simple yet highly effective, as it shows good structural encoding capability (Sec 4.1), shape bias (even without training, Sec 4.2) and parts/subparts learning dynamics (Sec 4.5). Finally the operation (with training) is validated with tasks including style robustness (ResNet-18, Sec 4.3) and few-shot image synthesis (FastGAN, Sec 4.4) on ImageNet subsets, where it also shows promising results.

Strengths

+ [Originality] The paper is sufficiently novel in my opinion. Although the idea of top-k sparsity has been commonly used in various applications (training/inference acceleration, brain-inspired algorithms, etc.), the fact that the simple idea works for this new, challenging topic should be interesting to the community. + [Clarity] The paper is clear and easy to follow. The effort to clearly explain the motivations (e.g. Sec 4.4) and results (e.g. Sec 4.5) of the experiments is relatively noteworthy. + [Quality] The paper is overall of good quality in my opinion. The proposed method is well motivated, simple and effective. The evaluation, though purely empirical, is relatively comprehensive and promising. + [Significance] Given the simplicity and effectiveness of the proposed method, and the relevance of the topic, I think this work is significant and could be valuable to many in the community.

Weaknesses

- [Evaluation, Minor] To further improve the impact of the paper, the authors could consider improving the following aspects. 1) Fig 2 and A.1 are both valuable yet limited. The authors could consider including larger-scale experiments/visualization and quantitative analysis of the results (e.g. textureness [38]). 2) The authors could consider including experiments combining with existing solutions (e.g. data augmentation based, given the orthogonality) on the benchmark [10, 39] to demonstrate this work’s potential to further boost performance, including on ViT-based architectures. [38] The Synthesizability of Texture Examples, CVPR, 2014.\ [39] https://github.com/bethgelab/model-vs-human/

Questions

1) Does the value of non-top-k units (now set to 0 in Eq 1) affect the experimental results (e.g. by overly shifting the mean or other statistics)? Shouldn’t some form of corrective offset (or learnable value other than 0) be applied? 2) How does the top-k without training performance compare to others in Table 1? 3) What do the colors mean in Fig 7?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

3 good

Contribution

3 good

Limitations

The proposed method itself doesn’t seem to have serious limitations as far as I can tell. However, the limitations of the standard protocols & datasets [10] currently used to analyze the results could be discussed in the paper.

Summary

Current deep learning models are known to have more texture bias for object recognition tasks while humans have more shape bias. They test the hypothesis that sparse coding may improve shape bias in deep learning models. Using Top-K operations, they show improved accuracy in pre-trained vison models as well as improved shape bias in models trained from scratch. Furthermore, in few-shot image systhesis tasks, models with sparsity constraints result in more structurally coherent images. The Top-K operations may be imposing binary masking on object parts, which can induce shape bias.

Strengths

- Connecting sparsity with shape bias is novel. - The authors examine both classification and generative models based on the sparsity operations. - Their study has implicaitons in both building computational models and understanding human vision. Their empirical results suggest that sparsity can lead to improved shape bias, making the models potentially make judgments closer to humans. Conversely, their study suggests sparse encodings may play a role in inducing shape bias in human vision. - Presentation is clear with helpful figures. I enjoyed reading it.

Weaknesses

- Section 4.1: Additional comparisons with Ts w. Top-K only will be useful. It is unclear whether less structural information in TS w. Non Top-K is just due to a lack of representational power. - Section 4.2: As noted in line 160, Vision Transformers show improved shape bias. How would you apply Top-K operations to Vision Transformers and do you expect to obtain more shape bias for those models? Since they are a popular model family, I think it's worth adding experiments or adding notes on them. - Section 4.3: Does the Top-K training lead to many dead neurons, essentially reducing the network size, which would work as an implicit regularization? - Section 4.5: The idea suggested here seems related to models developed with topographic constraints (Margalit et al., 2023), where spatially smooth response patterns are learned. Comments on these models will be useful. - Section 4.6: The discussion here implies that an ideal sparsity rate may be different for each layer. Applying different K values depending on layer depth or not applying sparsity to some layers in the experiments (e.g. Figure 3) will be interesting to test the hypothesis further. - While I do think the study suggests an interesting perspective on shape bias using sparsity, the study may have less impact if models trained on larger datasets show more shape bias. For instance, in Figure 3, I wonder ConvNexts or ViTs trained on LAION-2B would give a stronger baseline. Minor comments - Line 53 ("indeed can indeed introduce"): indeed is repeated - Line 161 ("already can already induced"): already is repeated - Figure 4: I assumed it indicates humans, but the label for purple markers is missing.

Questions

- Section 4.2: How does the number of correct recognitions change when sparsity is imposed on the models? Are we sacrificing accrucy a lot for better shape bias? - Figure 3: For the boxplots, please explain the distribution, whether it is over multiple image categories or multiple seeds for applying sparsity, etc.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

Currently, the study tested only convolutional architectures. Otherwise, limitations are adequately addressed.

Summary

This paper proposes enforcing sparsity within a neural network through a top-K mechanism and demonstrates that this leads in an emergent fashion an emphasis on shape rather texture in the learned representations of the network. This in turn leads to improvements in robustness for network behaviour.

Strengths

- The paper is clearly written and laid out. - The paper presents a novel and innovative contribution to a challenging goal in deep learning (namely, shifting neural networks more toward shape representations rather than texture representations). Although this topic has been studied before, as the paper points out existing methods to achieve this have significant drawbacks (such as the practicality of style-transfer augmentations), whereas the method proposed in this paper is extremely practical to implement. - The use of multiple different styles of experimental methods helps round out the conclusions (for example, Section 4.1 is a relatively straightforward qualitative experiment that is nevertheless compelling, naturally leading to the more comprehensive later experiments. -

Weaknesses

- I found it wasn't always clear whether Top-K was being applied to all layers or a subset of layers. Section 4.4 explicitly mentions that Top-K was used only in the fourth layer; why this layer, and how was this determined?

Questions

- In Figure 6(b) is there a difference between Ours 1 and Ours 2, or is that simply two different examples of output? - Does the use of a top-K mechanism lead to some of the network essentially becoming redundant (i.e. always ignored) and wasted computation?

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

4 excellent

Limitations

The paper does not have any obvious limitations that were not discussed.