An Interpretable Probabilistic Approach for Demystifying Black-box Predictive Models

The use of sophisticated machine learning models for critical decision making\nis faced with a challenge that these models are often applied as a "black-box".\nThis has led to an increased interest in interpretable machine learning, where\npost hoc interpretation presents a useful mechanism for generating\ninterpretations of complex learning models. In this paper, we propose a novel\napproach underpinned by an extended framework of Bayesian networks for\ngenerating post hoc interpretations of a black-box predictive model. The\nframework supports extracting a Bayesian network as an approximation of the\nblack-box model for a specific prediction. Compared to the existing post hoc\ninterpretation methods, the contribution of our approach is three-fold.\nFirstly, the extracted Bayesian network, as a probabilistic graphical model,\ncan provide interpretations about not only what input features but also why\nthese features contributed to a prediction. Secondly, for complex decision\nproblems with many features, a Markov blanket can be generated from the\nextracted Bayesian network to provide interpretations with a focused view on\nthose input features that directly contributed to a prediction. Thirdly, the\nextracted Bayesian network enables the identification of four different rules\nwhich can inform the decision-maker about the confidence level in a prediction,\nthus helping the decision-maker assess the reliability of predictions learned\nby a black-box model. We implemented the proposed approach, applied it in the\ncontext of two well-known public datasets and analysed the results, which are\nmade available in an open-source repository.\n