Know Your Limits: Uncertainty Estimation with ReLU Classifiers Fails at Reliable OOD Detection

A crucial requirement for reliable deployment of deep learning models for\nsafety-critical applications is the ability to identify out-of-distribution\n(OOD) data points, samples which differ from the training data and on which a\nmodel might underperform. Previous work has attempted to tackle this problem\nusing uncertainty estimation techniques. However, there is empirical evidence\nthat a large family of these techniques do not detect OOD reliably in\nclassification tasks.\n This paper gives a theoretical explanation for said experimental findings and\nillustrates it on synthetic data. We prove that such techniques are not able to\nreliably identify OOD samples in a classification setting, since their level of\nconfidence is generalized to unseen areas of the feature space. This result\nstems from the interplay between the representation of ReLU networks as\npiece-wise affine transformations, the saturating nature of activation\nfunctions like softmax, and the most widely-used uncertainty metrics.\n