Prior choice affects ability of Bayesian neural networks to identify unknowns

Deep Bayesian neural networks (BNNs) are a powerful tool, though\ncomputationally demanding, to perform parameter estimation while jointly\nestimating uncertainty around predictions. BNNs are typically implemented using\narbitrary normal-distributed prior distributions on the model parameters. Here,\nwe explore the effects of different prior distributions on classification tasks\nin BNNs and evaluate the evidence supporting the predictions based on posterior\nprobabilities approximated by Markov Chain Monte Carlo sampling and by\ncomputing Bayes factors. We show that the choice of priors has a substantial\nimpact on the ability of the model to confidently assign data to the correct\nclass (true positive rates). Prior choice also affects significantly the\nability of a BNN to identify out-of-distribution instances as unknown (false\npositive rates). When comparing our results against neural networks (NN) with\nMonte Carlo dropout we found that BNNs generally outperform NNs. Finally, in\nour tests we did not find a single best choice as prior distribution. Instead,\neach dataset yielded the best results under a different prior, indicating that\ntesting alternative options can improve the performance of BNNs.\n