A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challenges

Machine learning models often encounter samples that are diverged from the\ntraining distribution. Failure to recognize an out-of-distribution (OOD)\nsample, and consequently assign that sample to an in-class label significantly\ncompromises the reliability of a model. The problem has gained significant\nattention due to its importance for safety deploying models in open-world\nsettings. Detecting OOD samples is challenging due to the intractability of\nmodeling all possible unknown distributions. To date, several research domains\ntackle the problem of detecting unfamiliar samples, including anomaly\ndetection, novelty detection, one-class learning, open set recognition, and\nout-of-distribution detection. Despite having similar and shared concepts,\nout-of-distribution, open-set, and anomaly detection have been investigated\nindependently. Accordingly, these research avenues have not cross-pollinated,\ncreating research barriers. While some surveys intend to provide an overview of\nthese approaches, they seem to only focus on a specific domain without\nexamining the relationship between different domains. This survey aims to\nprovide a cross-domain and comprehensive review of numerous eminent works in\nrespective areas while identifying their commonalities. Researchers can benefit\nfrom the overview of research advances in different fields and develop future\nmethodology synergistically. Furthermore, to the best of our knowledge, while\nthere are surveys in anomaly detection or one-class learning, there is no\ncomprehensive or up-to-date survey on out-of-distribution detection, which our\nsurvey covers extensively. Finally, having a unified cross-domain perspective,\nwe discuss and shed light on future lines of research, intending to bring these\nfields closer together.\n