Modeling the Distribution of Normal Data in Pre-Trained Deep Features for Anomaly Detection

Anomaly Detection (AD) in images is a fundamental computer vision problem and\nrefers to identifying images and image substructures that deviate significantly\nfrom the norm. Popular AD algorithms commonly try to learn a model of normality\nfrom scratch using task specific datasets, but are limited to semi-supervised\napproaches employing mostly normal data due to the inaccessibility of anomalies\non a large scale combined with the ambiguous nature of anomaly appearance.\n We follow an alternative approach and demonstrate that deep feature\nrepresentations learned by discriminative models on large natural image\ndatasets are well suited to describe normality and detect even subtle anomalies\nin a transfer learning setting. Our model of normality is established by\nfitting a multivariate Gaussian (MVG) to deep feature representations of\nclassification networks trained on ImageNet using normal data only. By\nsubsequently applying the Mahalanobis distance as the anomaly score we\noutperform the current state of the art on the public MVTec AD dataset,\nachieving an AUROC value of $95.8 \\pm 1.2$ (mean $\\pm$ SEM) over all 15\nclasses. We further investigate why the learned representations are\ndiscriminative to the AD task using Principal Component Analysis. We find that\nthe principal components containing little variance in normal data are the ones\ncrucial for discriminating between normal and anomalous instances. This gives a\npossible explanation to the often sub-par performance of AD approaches trained\nfrom scratch using normal data only. By selectively fitting a MVG to these most\nrelevant components only, we are able to further reduce model complexity while\nretaining AD performance. We also investigate setting the working point by\nselecting acceptable False Positive Rate thresholds based on the MVG\nassumption.\n Code available at https://github.com/ORippler/gaussian-ad-mvtec\n