Do Saliency Models Detect Odd-One-Out Targets? New Datasets and Evaluations

Recent advances in the field of saliency have concentrated on fixation\nprediction, with benchmarks reaching saturation. However, there is an extensive\nbody of works in psychology and neuroscience that describe aspects of human\nvisual attention that might not be adequately captured by current approaches.\nHere, we investigate singleton detection, which can be thought of as a\ncanonical example of salience. We introduce two novel datasets, one with\npsychophysical patterns and one with natural odd-one-out stimuli. Using these\ndatasets we demonstrate through extensive experimentation that nearly all\nsaliency algorithms do not adequately respond to singleton targets in synthetic\nand natural images. Furthermore, we investigate the effect of training\nstate-of-the-art CNN-based saliency models on these types of stimuli and\nconclude that the additional training data does not lead to a significant\nimprovement of their ability to find odd-one-out targets. Datasets are\navailable at http://data.nvision2.eecs.yorku.ca/P3O3/.\n