Summary
This paper explores the relationship between the alignment of deep learning based representations with humans and the downstream performance of the NN models. Using triplets as a non-metric measure of similarity, the authors use information theory to show that alignment and downstream performance would have a U-shaped relationship, where low and high alignment would give a good downstream performance.
Strengths
* The authors use existing ideas of triplets from cognitive science and combine information theory to show why human alignment may not always lead to better performance, especially for few-shot downstream learning. More importantly, the authors give an intuition and theoretical backing for why some models can have low alignment with humans and still perform well in downstream learning.
* The authors highlight the significance of ensuring human alignment of neural network representations by studying the role of alignment on two critical characteristics: adversarial robustness and domain shift robustness. They further show how human alignment plays a role in downstream learning, mainly when limited data is available.
* Using a significantly large number of models and large-scale existing human-annotated data, the authors empirically validated the theoretical claims made in the paper, showing the benefit of very high alignment with robustness while also revealing the U-shaped relationship between few-shot learning and alignment. The authors performed critical statistical testing to highlight the results.
* The interaction of architecture model choices and alignment is insightful and could be helpful for future work.
Weaknesses
* The contribution seems limited, the main contribution seems to be the showing of the U-shaped relationship. The primary notion of triplets is derived from prior work.
* The importance of triplets seems limited. Pearson and Spearman also clearly provide similar takeaways. Why are triplets needed? Not clear from the evaluation section.
* Evaluation is also limited in some aspects, although the paper explores more than 400 models. Most transfer learning results are shown for CIFAR after ImageNet pre-training. The result on MNIST and FMNIST show quite weak results, meaning that strong domain shift during transfer learning makes the metric less impactful. But transfer learning is usually most important in these settings (e.g., medical domain, face classification, etc.).
* There is a lack of clarity in the writing of the paper. Section 3 is difficult to follow, and it is not always clear how the different definitions flow, and how the pipeline works. E.g., not clear what is the significance of Def. 3.4, Discussion section mentions "quadruplets", but they are never mentioned in the paper. Not clear at all.
* Adversarial robustness is measured for real-world adversarial images. More common adversarial examples used in literature, e.g., $\ell_p$ ball perturbations are not explored.
* Evaluation section compares the triplets with pairwise measures like Pearson and Spearman correlations. But it is not clear from Section 3 as to why it is interesting to compare triplets to pairwise. Do triplets give benefits over Pearson/Spearman? Are there any tradeoffs?
* Results shown in the tables are not well discussed
* Transfer learning results only analyzed when we have "head" networks. Not clear if the method works for full network fine-tuning or not, something that is often used in practice.
* No justification given on the choice of the heads, e.g., dropout rate, etc. Were they hyper parameter-tuned?
* Not clear what and how the ImageNet accuracy correction is performed. It seems important, so more intuition should be provided.
Questions
* Why does correlation with test accuracy reduce with more training examples (Fig. 3 and 4 right)?
* How are the classifier heads parameters chosen?
* What is meant by supervised learning in Section 3? Does it mean standard classification task? It is not clear.
* What are quadruplets mentioned in Discussion?
* Large datasets like ImageNet and CIFAR have a lot of "noisy" and misclassified examples on which the deep models are trained on. Do the different alignment metrics reveal any interesting facts on the model's alignment or non-alignment on these "outlier" examples?
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Limitations
* Most transfer learning results are shown for transfer from ImageNet to CIFAR, which is limiting.
* Exact contribution of triplets is not clear from the paper, as existing correlation metrics can already show the U-shaped relation and performance relations.