A statistical framework for efficient out of distribution detection in deep neural networks

Background. Commonly, Deep Neural Networks (DNNs) generalize well on samples\ndrawn from a distribution similar to that of the training set. However, DNNs'\npredictions are brittle and unreliable when the test samples are drawn from a\ndissimilar distribution. This is a major concern for deployment in real-world\napplications, where such behavior may come at a considerable cost, such as\nindustrial production lines, autonomous vehicles, or healthcare applications.\nContributions. We frame Out Of Distribution (OOD) detection in DNNs as a\nstatistical hypothesis testing problem. Tests generated within our proposed\nframework combine evidence from the entire network. Unlike previous OOD\ndetection heuristics, this framework returns a $p$-value for each test sample.\nIt is guaranteed to maintain the Type I Error (T1E - incorrectly predicting OOD\nfor an actual in-distribution sample) for test data. Moreover, this allows to\ncombine several detectors while maintaining the T1E. Building on this\nframework, we suggest a novel OOD procedure based on low-order statistics. Our\nmethod achieves comparable or better results than state-of-the-art methods on\nwell-accepted OOD benchmarks, without retraining the network parameters or\nassuming prior knowledge on the test distribution -- and at a fraction of the\ncomputational cost.\n