On the Agreement of Deep Neural Networks with the Brain in Encoding Visual Stimuli: Implications for Image Quality Assessment

Abstract

Deep neural networks (DNNs) have performed remarkably in various computer vision tasks. Considering the great success of DNNs and their inspiration from biological vision, it is interesting to explore the extent to which these frameworks can reflect the brain responses in the visual cortex. In this paper, we first evaluate the effectiveness of different DNN models in predicting brain neural activities when observing natural images. Next, we examined how the brain-like performance of deep features degrades under image distortion and whether such degradation is aligned with human quality preferences. The outcomes reveal that although DNN models are still far from optimal in modeling brain responses, some models can represent a hierarchical data processing scheme in accordance with the visual cortex. In addition, deep features showed good consistency with the human opinion on the degradation impact of different distortion types, making them good candidates for the design of biologically-inspired image quality assessment models.