What Do Deep Saliency Models Learn about Visual Attention?

Summary

This paper proposes a new framework, which can decompose the learned features of a saliency model into trainable bases (using [42]), those bases are combined to formulate the final saliency map, and the weight of the combination indicates the contribution of each basis. The semantic meaning of each basis can be explored by matching the bases to a probe dataset (Visual Genome is used in this draft), Thus, saliency can be studied based on visual concepts, .e.g, different types of visual relationship, or objects, exploring what the key factors are behind the saliency region.

Strengths

This draft is well written, the organization is clear, and the method is explained in details. The proposed method attempts to solve a fundamental and important question, what features are important to saliency.

Weaknesses

I like this study attempts to solve a very interesting question, however, some of the designs may need more discussion. This study is based on the existing basis de-composition[42] technique. The main contribution is to link those bases to other visual concepts by computing thresholded IOU. This design heavily relies on the probing dataset, which makes the hyper-parameter threshold a little tricky. What if other datasets are used, will the bases also match other concepts, how one can validate the matched concept (objects or events) is valid. Some studies attempt to explore the learned feature based on known probing concepts, e.g., designing a dataset specifically for texture and shape concepts(Geirhos, Robert, et al. 2018). Exploring visual concepts (bases) by matching to other datasets is not very convincing. Should one exhaustively match to all concepts to find the best match (I highly doubt if this is feasible or how to define the universe), or setting a threshold and matching to a “local”’ solution(a specific dataset)? Thus, matching bases to a specific dataset is not very solid to show what the bases are. More generally, assuming we can find the “ground-truth” (the true meaning of those bases), I believe saliency is way more complicated than simply based on visual concepts. From Figure 2, we can see that street sign is less important than faces. We can imagine the street sign could be the most salient region if the picture only contains a street sign in the middle. Given two images, one has a face in the middle and one has a street sign in the middle, they could be equally salient. Thus, one explanation for figure 2 could also be it shows the bias of the dataset which used to train the model, more faces appeared in that dataset. There also could be biases in the probe dataset, Visual Genome in this study. One concept, say “having meeting”, is more salient could be the visual features happen to match one of the bases. The real most salient basis does not exist in that dataset. Another problem in this study is that saliency may not depend on the semantic. Imagine we have ten faces in a picture, the middle could be the most salient, while the others are not. Thus, study which visual basis is the most salient is not convincing. This may also happen to the Visual Genome dataset. The region for the visual relation is the union of the head and the tail, which is larger than the two objects. The bigger union region may better correlate with the well-known centre bias regardless of what the actual content. This is why “action” is the most salient in all of the datasets in Figure 3. Overall, applying basis decomposition [42] is not new, the finding of the semantic importance given saliency may simply showing dataset biases. Geirhos, Robert, et al. "ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness." arXiv preprint arXiv:1811.12231 (2018).

Questions

See above

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

Exploring unknown bases to a dataset is questionable, which requires further study. The finding of those bases may not explain saliency, but the biases of the training and probing datasets.

Summary

This paper examines the problem of predicting visual saliency in images. Unlike many other works, it focuses on determining what leads to the predictions made, including underlying features that are learned, and formulating the prediction as a combination of bases. Through this, one is able to garner an understanding of how the models make their decisions and attach this to semantic information or other concepts.

Strengths

Strengths of this paper are as follows: 1. It approaches saliency from a different point of view; rather than competing to marginally beat an ROC score, it takes a step towards truly understanding the success of deep learning models in this domain 2. By explicitly modelling features correlated with attention, and those inversely correlated with attention, one can model both what is salient and what is not in a way that allows these concepts to be quantified 3. Because the model constructs it's prediction from a set of bases, this effectively allows human attention to be modelled subject to a change to these bases, as demonstrated in an example involving autism.

Weaknesses

Ultimately, it would be nice to see some quantitative results on how well the model performs compared to some of the state-of-the-art models, although I understand this is not the point of the paper. Nevertheless, a detailed set of results along this dimension could add value but I wouldn't insist on it.

Questions

1. If possible, can you comment on how this formalism for saliency prediction extends to other domains? 2. In figure 5, it seems that a single epoch of fine tuning is closer to careful fine tuning than the original weights. Is this from using a relatively large learning rate? 3. Related to the above, how does this vary with changes to the learning rate?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors have addressed limitations of their approach in Section 5 of the paper.

Summary

This paper attempts to decompose the learned representation of a data-driven saliency model into a constituent set of bases that are mapped onto semantic concepts, thereby providing insight into what is driving the model's representation of saliency. This method is applied to three different saliency models of varying formulation over several datasets. Some discussion is then provided of the results, as well as some qualitative discussion of examples of model failures.

Strengths

The paper takes on a challenging and open-ended problem in saliency modelling, namely the difficulty of teasing apart the different contributions to attentional capture. Similarly, the work provides an example of continued work in general explainability within deep learning methods, which is an important issue within the field. The paper is clearly written and relatively easy to follow, although there are a few details that are missing (such as the choice of N).

Weaknesses

There are some references that seem pertinent that were not discussed. In particular, significant work on failure modes of modern saliency models was poorly represented. - Bruce et al., "A Deeper Look at Saliency: Feature Contrast, Semantics, and Beyond", CVPR 2016 -- This paper digs into some of the failure modes common to deep learning models, including object vs. background and semantic vs. feature contrast elements (e.g. see Figure 7), providing pertinent insights to the discussion in the submission. - Kümmerer et al., "Understanding Low- and High-Level Contributions to Fixation Prediction", ICCV 2017 -- This paper explicitly explores constituent elements of saliency representation in deep networks from the perspective of high-level vs. low-level features. Given the way the paper attempts to tease apart the representation of saliency between different feature classes, it is conceptually highly relevant background for the current submission. - Kotseruba et al., "Do Saliency Models Detect Odd-One-Out Targets? New Datasets and Evaluations", BMVC 2019 -- This paper provides a psychophysical (P^3) and natural image (O^3) dataset with targets explicitly defined by low-level salient features (e.g. colour, orientation, shape, or size singletons), and finds that saliency models (including deep learning-based models) largely perform quite poorly. For exploring failure rates the O^3 dataset would be a potentially useful (albeit ground truth was defined by semantic object annotation and not fixation data), but even if the dataset is not used the examination of model failures in the submission should include the context of this prior exploration. - Tatler et al., "Visual correlates of fixation selection: effects of scale and time", Vision Research, 2005 -- This paper explores the evolution of fixations through time, including aspects such as central vs. peripheral distribution and inter-subject consistency of fixation location. Given that this is one area that the submission claims novelty, it would be good to put it in context with prior explorations in this area of the temporal evolution of low-level human attention. Overall, while the paper is interesting and tackles an exceedingly challenging problem, I think there are a number of conceptual issues that it needs to overcome. While some specific issues are given in the questions below, the primary issue is that while the submission encodes the positive/negative importance of the various bases extracted, it is well established within psychophysics that the relative importance of elements to saliency is contextual (e.g. see Nothdurft (1993) "Saliency effects across dimensions in visual search"; Nothdurft (2000) "Salience from feature contrast: additivity across dimensions" for some low level examples), and so these relative attributes are likely to change from image to image. Within the current submission, these attributes change from model to model and dataset to dataset; what conclusions are to be drawn from this? Is the technique shedding light on dataset composition, model bias, or some deeper aspect of relative aspects of saliency? Much of the analysis is presented without clear connection to either human behaviour (with the exception of Section 4.5, which cleverly makes use of the technique to explore the representations learned from data from different human subject populations or conditions) or model performance in a traditional sense, making it difficult to put into context or derive deeper insight.

Questions

- Although drawing on the architectures of SALICON, DINet, and TranSalNet, the model configurations in this paper are distinct from the instantiations of the original publications in order to accommodate the need for trainable bases. How does that change the ultimate behaviour of the models with respect to standard measures of saliency performance? Alternatively, even a quantified value for the change in saliency maps when compared within model (e.g. the paper's version of SALICON correlated against the standard instantiation of SALICON) would help put the paper's results in context with the existing literature. - What is the value of N (the number of bases)? How stable are the results with respect to N (i.e. does the mapped semantic content change substantially with even small changes in N)? - Each basis is mapped onto a top-5 semantic mix. Why not onto a single concept? How was 5 selected? Similar to the previous question, how does this choice affect the subsequent analysis? - How is "action" defined in a static image? The primary example given, "having meeting", seems like a social activity, which was a separate category. I get that it is challenging to relate the messy details of semantic categorization in a short paper, but given that this is central to the topic of the paper, I think it needs a clearer explanation. - How do the insights provided in this paper relate to model performance? When the models show a markedly different breakdown of salient factors (e.g. Figure 4, which shows SALICON emphasizing vehicles much more strongly than TranSalNet, while TranSalNet emphasizes clothing more than any other model), does this correlate with predictive accuracy? - Related to the previous question, could you use the IoU Measurement process used to assign labels to the bases to provide an approximate breakdown of the factors leading to human fixations directly? This might provide another point of comparison to better put the results of this paper in context. - Given the range of behaviours across the models shown in Figure 4, why does Figure 5 average the semantic weights across models (also, this should be noted in the caption; when I first read the paper I was quite confused which model was being shown in Figure 5)? What is the justification for this? Do the models tend to converge after fine-tuning? - Figure 6 (b.) and (c.): are these results for subjects without autism? If so, this should be more clearly noted. - What was meant by line 304: "Whether models... open question."? What would lead to a model behaving "even better than humans", given that humans are the system trying to be modelled?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

The limitations discussed seem clear.

Summary

The paper presents a novel analytic framework that provides a principled interpretation and quantification of the implicit features learned by deep saliency models, which are used for predicting human visual attention. The framework decomposes these features into interpretable bases aligned with semantic attributes, and reformulates saliency prediction as a weighted combination of probability maps. The authors conducted extensive analyses to understand the factors contributing to the success of saliency models, including the positive and negative weights of semantics, the impact of training data and architectural designs, and the effects of fine-tuning. They also explored visual attention in various application scenarios, such as autism spectrum disorder, emotion-eliciting stimuli, and attention evolution over time. The study identifies the accurate feature detection and differentiation of semantics as key factors in the models' success, influenced by training data and design choices. The framework is also useful for characterizing human visual attention and understanding common failure patterns in saliency models. The authors suggest incorporating structures and lower-level information for improved modeling. The research has potential impacts on optimizing human-computer interfaces, assisting visually impaired individuals, and enhancing societal benefits.

Strengths

1. The paper presents a novel analytic framework that provides interpretation and quantification of the implicit features learned by deep saliency models for predicting human visual attention. The framework decomposes implicit features into interpretable bases aligned with semantic attributes, allowing for a weighted combination of probability maps connecting the bases and saliency in saliency prediction. The framework effectively identifies a variety of semantics learned by deep saliency models, including social cues, actions, clothing, and salient object categories, showcasing its versatility in analyzing attention across diverse scenarios. 2. The framework reveals the positive and negative contributions of semantics to saliency, highlighting the ability of deep saliency models to distinguish between salient and non-salient semantics. 3. The analysis demonstrates how training data and model designs impact saliency prediction, with shifts in semantic weights reflecting the characteristics of the datasets and models. 4. The study also investigates the effects of fine-tuning on semantic weights, showing how deep saliency models progressively adapt features during training to better capture salient cues and refine the weights of negative semantics. 5. The framework is also applied to explore the capture of human attention characteristics, such as the impact of visual preferences, characteristics of visual stimuli (e.g., emotions), and temporal dynamics, providing insights into the factors influencing attention deployment. 6. The findings validate the effectiveness of deep saliency models in automatically identifying salient semantics, differentiating foreground from background, and encoding fine-grained characteristics of attention.

Weaknesses

The work has a lot of novelty with good empirical analyses. The paper is also nicely written. However, I find that the authors have missed comparing the performance of their model with other saliency models using standard saliency metrics as can be found in this leaderboard: http://saliency.mit.edu/results_mit300.html. The authors have also failed to cite some important works from like DeepGaze, DeepFix, BMS, etc.

Questions

I am curious to know if the authors could perform and share the results of comparison with existing methods using the standard metrics for measuring saliency.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors analyzed the failure patterns of deep saliency models within the intermediate inference process using their proposed factorization framework. They select common success and failure examples where three tested models consistently have high/low NSS scores. They then perform a qualitative analysis by visualizing the spatial probabilistic distribution of the bases for semantics with positive and negative weights. In the successful examples, they find that accurate saliency prediction is correlated with the differentiation of diverse semantics. The stimuli in these examples typically have salient and non-salient regions belonging to different semantics. Therefore, the models, with the ability to distinguish positive and negative semantics, can readily determine the saliency distribution. On the other hand, in the failure examples, the models struggle to determine saliency within objects or among objects with similar semantics. Investigation of the probabilistic distribution of bases reveals that models often have a uniform-like distribution of bases on object parts or among objects of the same category, making it difficult to construct accurate saliency maps. The analysis also highlights that existing models have difficulty with scenes without salient objects. These scenes are challenging for the models as they lack clear focal points. The ground truth human attention in these failure patterns exhibits high inter-subject variability, suggesting that human viewers may not agree on where to look, making the ground truth maps less reliable. The authors raise the question of whether models may perform as well as or even better than humans in these challenging situations. Based on their observations, the authors hypothesize that leveraging more structured representations to encode contextual relationships between semantics and integrating mid- and low-level cues may be beneficial in addressing these failure patterns in certain scenarios. This suggests that incorporating additional information and incorporating contextual understanding may help improve the performance of deep saliency models.

feedback

Thanks the authors for the further explanation, however, not all of the questions are answered in detail. 1. " Visual Genome, the biggest and most diverse naturalistic dataset with a sufficiently large number of images". I looked into the VG dataset, which is also very skewed, 90% relations belong to the categories of possessive(has part of) and geometric(above behind). More importantly, "people" and "vehicles" are also dominant in the objects. Thus, it is difficult to say the VG dataset is "large" (or "fair"?) enough to test saliency. Moreover, it is unknown to me how to define a dataset is large enough for this purpose. This also explains the use of the adaptive threshold. The true unknown concept does not exist, so a lower threshold is used to select the second or third best match. 2.b "include diverse objects in naturalistic context; so the confounder effects such as center bias are neutralized to some degree,". I looked in to the VG data, I cannot see how the diverse objects can solve the centre bias effect. In contrast, the definition in visual scene graph could worsen this problem. The visual feature of relation, "having meeting", the union region of two objects correlates more with centre bias, which explains why the relationship is more salient, instead of reducing the center bias effect. 3. Thanks for the explanation, this could partially answer the frequency in the probing dataset. Could you please add a little more (maybe one sentence about this frequency normalization Z), is this new in this study? I might miss it in [43]. 4. Again, it is still unclear how to define VG is large enough for this purpose, and the relation (union region) definitely correlates more with the saliency region due to the center bias. 5. a) This size-intensive measure Eq.4 cannot handle the center bias problem. Imagining that most saliency appears more in the center of an image, the union region also has this tendency comparing with the head and tail objects. Thus, Eq4 could easily capture relation (union region) than single objects regardless of what threshold is applied. b). VG dataset is not balanced though it has a lot of categories. More importantly, the VG data also has centre bias due to the human bias, and the annotation (relation) is created by human where relation around RoIs(center) is labeled. Thus, both measures are not used for the position bias.