Explaining predictions of deep neural networks (DNNs) is an important and\nnontrivial task. In this paper, we propose a practical approach to interpret\ndecisions made by a DNN object detector that has fidelity comparable to\nstate-of-the-art methods and sufficient computational efficiency to process\nlarge datasets. Our method relies on recent theory and approximates Shapley\nfeature importance values. We qualitatively and quantitatively show that the\nproposed explanation method can be used to find image features which cause\nfailures in DNN object detection. The developed software tool combined into the\n"Explain to Fix" (E2X) framework has a factor of 10 higher computational\nefficiency than prior methods and can be used for cluster processing using\ngraphics processing units (GPUs). Lastly, we propose a potential extension of\nthe E2X framework where the discovered missing features can be added into\ntraining dataset to overcome failures after model retraining.\n