A Survey on Assessing the Generalization Envelope of Deep Neural Networks: Predictive Uncertainty, Out-of-distribution and Adversarial Samples

Deep Neural Networks (DNNs) achieve state-of-the-art performance on numerous\napplications. However, it is difficult to tell beforehand if a DNN receiving an\ninput will deliver the correct output since their decision criteria are usually\nnontransparent. A DNN delivers the correct output if the input is within the\narea enclosed by its generalization envelope. In this case, the information\ncontained in the input sample is processed reasonably by the network. It is of\nlarge practical importance to assess at inference time if a DNN generalizes\ncorrectly. Currently, the approaches to achieve this goal are investigated in\ndifferent problem set-ups rather independently from one another, leading to\nthree main research and literature fields: predictive uncertainty,\nout-of-distribution detection and adversarial example detection. This survey\nconnects the three fields within the larger framework of investigating the\ngeneralization performance of machine learning methods and in particular DNNs.\nWe underline the common ground, point at the most promising approaches and give\na structured overview of the methods that provide at inference time means to\nestablish if the current input is within the generalization envelope of a DNN.\n