Interpretable Machine Learning with an Ensemble of Gradient Boosting Machines

A method for the local and global interpretation of a black-box model on the\nbasis of the well-known generalized additive models is proposed. It can be\nviewed as an extension or a modification of the algorithm using the neural\nadditive model. The method is based on using an ensemble of gradient boosting\nmachines (GBMs) such that each GBM is learned on a single feature and produces\na shape function of the feature. The ensemble is composed as a weighted sum of\nseparate GBMs resulting a weighted sum of shape functions which form the\ngeneralized additive model. GBMs are built in parallel using randomized\ndecision trees of depth 1, which provide a very simple architecture. Weights of\nGBMs as well as features are computed in each iteration of boosting by using\nthe Lasso method and then updated by means of a specific smoothing procedure.\nIn contrast to the neural additive model, the method provides weights of\nfeatures in the explicit form, and it is simply trained. A lot of numerical\nexperiments with an algorithm implementing the proposed method on synthetic and\nreal datasets demonstrate its efficiency and properties for local and global\ninterpretation.\n