Anomaly detection through latent space restoration using vector-quantized variational autoencoders

We propose an out-of-distribution detection method that combines density and\nrestoration-based approaches using Vector-Quantized Variational Auto-Encoders\n(VQ-VAEs). The VQ-VAE model learns to encode images in a categorical latent\nspace. The prior distribution of latent codes is then modelled using an\nAuto-Regressive (AR) model. We found that the prior probability estimated by\nthe AR model can be useful for unsupervised anomaly detection and enables the\nestimation of both sample and pixel-wise anomaly scores. The sample-wise score\nis defined as the negative log-likelihood of the latent variables above a\nthreshold selecting highly unlikely codes. Additionally, out-of-distribution\nimages are restored into in-distribution images by replacing unlikely latent\ncodes with samples from the prior model and decoding to pixel space. The\naverage L1 distance between generated restorations and original image is used\nas pixel-wise anomaly score. We tested our approach on the MOOD challenge\ndatasets, and report higher accuracies compared to a standard\nreconstruction-based approach with VAEs.\n