Comparing object recognition in humans and deep convolutional neural networks -- An eye tracking study

Deep convolutional neural networks (DCNNs) and the ventral visual pathway\nshare vast architectural and functional similarities in visual challenges such\nas object recognition. Recent insights have demonstrated that both hierarchical\ncascades can be compared in terms of both exerted behavior and underlying\nactivation. However, these approaches ignore key differences in spatial\npriorities of information processing. In this proof-of-concept study, we\ndemonstrate a comparison of human observers (N = 45) and three feedforward\nDCNNs through eye tracking and saliency maps. The results reveal fundamentally\ndifferent resolutions in both visualization methods that need to be considered\nfor an insightful comparison. Moreover, we provide evidence that a DCNN with\nbiologically plausible receptive field sizes called vNet reveals higher\nagreement with human viewing behavior as contrasted with a standard ResNet\narchitecture. We find that image-specific factors such as category, animacy,\narousal, and valence have a direct link to the agreement of spatial object\nrecognition priorities in humans and DCNNs, while other measures such as\ndifficulty and general image properties do not. With this approach, we try to\nopen up new perspectives at the intersection of biological and computer vision\nresearch.\n