Bridging Local Interpretability: Analyzing SHAP Importance and Counterfactual Sensitivity for Model Transparency

AI systems are being widely adopted in decision processes in the field of healthcare, it has become imperative to ensure that there is transparency and interpretability in machine learning models. Several methods for explaining machine learning models have been proposed, with little being done to compare attribution-based methods with counterfactual-based methods for explaining machine learning models. In this study, we investigate the relationship between SHapley Additive exPlanations (SHAP) and Counterfactual Explanations for understanding machine learning interpretability from both descriptive and prescriptive perspectives. To achieve this, six machine learning models were trained on a heart disease dataset, with the top SHAP features being compared with counterfactual features. From the analysis, it is clear that counterfactuals can be used to effectively understand how sensitive the machine learning model is to feature variation, while SHAP provides stable feature importance across all machine learning models. Additionally, non-linear models like XG-Boost and Neural Networks have higher differences, while linear models like Logistic Regression have higher similarities in SHAP feature importance and counterfactual feature variations. These results demonstrate that combining SHAP-based attribution with counterfactual sensitivity analysis provides a complete and reliable approach to XAI, which can improve interpretability as well as decision support in real-world applications.