Ramifications of Approximate Posterior Inference for Bayesian Deep Learning in Adversarial and Out-of-Distribution Settings

Deep neural networks have been successful in diverse discriminative\nclassification tasks, although, they are poorly calibrated often assigning high\nprobability to misclassified predictions. Potential consequences could lead to\ntrustworthiness and accountability of the models when deployed in real\napplications, where predictions are evaluated based on their confidence scores.\nExisting solutions suggest the benefits attained by combining deep neural\nnetworks and Bayesian inference to quantify uncertainty over the models'\npredictions for ambiguous datapoints. In this work we propose to validate and\ntest the efficacy of likelihood based models in the task of out of distribution\ndetection (OoD). Across different datasets and metrics we show that Bayesian\ndeep learning models on certain occasions marginally outperform conventional\nneural networks and in the event of minimal overlap between in/out distribution\nclasses, even the best models exhibit a reduction in AUC scores in detecting\nOoD data. Preliminary investigations indicate the potential inherent role of\nbias due to choices of initialisation, architecture or activation functions. We\nhypothesise that the sensitivity of neural networks to unseen inputs could be a\nmulti-factor phenomenon arising from the different architectural design choices\noften amplified by the curse of dimensionality. Furthermore, we perform a study\nto find the effect of the adversarial noise resistance methods on in and\nout-of-distribution performance, as well as, also investigate adversarial noise\nrobustness of Bayesian deep learners.\n