Performance Analysis of Out-of-Distribution Detection on Various Trained Neural Networks

Several areas have been improved with Deep Learning during the past years.\nFor non-safety related products adoption of AI and ML is not an issue, whereas\nin safety critical applications, robustness of such approaches is still an\nissue. A common challenge for Deep Neural Networks (DNN) occur when exposed to\nout-of-distribution samples that are previously unseen, where DNNs can yield\nhigh confidence predictions despite no prior knowledge of the input.\n In this paper we analyse two supervisors on two well-known DNNs with varied\nsetups of training and find that the outlier detection performance improves\nwith the quality of the training procedure. We analyse the performance of the\nsupervisor after each epoch during the training cycle, to investigate\nsupervisor performance as the accuracy converges. Understanding the\nrelationship between training results and supervisor performance is valuable to\nimprove robustness of the model and indicates where more work has to be done to\ncreate generalized models for safety critical applications.\n