Selective Explanations

Summary

The paper presents a novel framework for explaining black box models through a selective explainer. The selective explainer unifies two different explanation methods: the former relies on amortized explainers, which are easier to compute but provide lower-quality explanations; the latter can provide higher-quality explanations but is more costly to compute. By identifying when the first method is enough to have a good explanation, the selective explainer can trade off (depending on the user’s needs) when relying on amortized explainers or a more complex explanation method. Experiments on benchmark data show the effectiveness of the approach.

Strengths

1. The paper considers an interesting problem, i.e. how to combine cheap-to-obtain explanations and expensive-to-obtain ones; 2. The theoretical analysis seems sound.

Weaknesses

w1. The methodology is based on never-discussed assumptions, i.e. a more uncertain/less stable explanation is a low-quality explanation. I think the authors should discuss this assumption further. Moreover, I guess such a measure is potentially correlated with whether we explain correct or incorrect ML predictions, which is something that is never discussed/taken into account and lacks appropriate references. w2. The terminology is misleading: the paper calls expensive-to-obtain explanations high-quality explanations. I think this misleads the reader because there is never an actual comparison between expensive explanations and their actual quality in explaining the underlying ML model. w3. The experimental evaluation is limited to quantitative measures. I think the paper would greatly benefit from including a user study, as humans must evaluate the quality of explanations provided by the novel selective explainer. For example, [Longo et al., 2024] provide a few reasons why human user studies should always be conducted when considering XAI methods. w4. The paper does not take into account relevant related work on uncertainty quantification of explanation methods. For instance, see [Zhao et al, 2021] and [Slack et al., 2021]. [Longo et al., 2024 ] Longo, L., Brcic, M., Cabitza, F., Choi, J., Confalonieri, R., Del Ser, J., Guidotti, R., Hayashi, Y., Herrera, F., Holzinger, A. and Jiang, R., 2024. Explainable artificial intelligence (XAI) 2.0: A manifesto of open challenges and interdisciplinary research directions. Information Fusion, p.102301. [Slack et al, 2021] Dylan Slack, Anna Hilgard, Sameer Singh, Himabindu Lakkaraju: Reliable Post hoc Explanations: Modeling Uncertainty in Explainability. NeurIPS 2021: 9391-9404 [Zhao et al., 2021] Xingyu Zhao, Wei Huang, Xiaowei Huang, Valentin Robu, David Flynn: BayLIME: Bayesian local interpretable model-agnostic explanations. UAI 2021: 887-896

Questions

Q1 - What is a low-quality explanation under the proposed framework? I think this is the core question underlying the work. However, the authors are not directly addressing this question, as, in my opinion, assessing the low quality of a certain explanation requires defining who is the final user of the explanation. Q2 - As far as I know, an unstable explanation might be due to the underlying uncertainty around a prediction, but also due to the approximations required to compute the explanation. Is there a way to disentangle these two kinds of uncertainty? I think this is something that the method should take into consideration. Q3—As discussed in W2, what the authors call a high-quality explanation seems to be obtained by a more costly (as it relies on some exact computation) explanation method. Is my understanding correct? If so, how do you ensure that the high quality explanation is a good explanation of why a certain prediction has been made? A few other details: Line 80: you refer to explanations with initial guesses. I would personally prefer to have the definition here rather than in the next section. Lines 152-154: can you clarify and discuss the drawbacks of this choice?

Rating

4

Confidence

4

Soundness

2

Presentation

2

Contribution

2

Limitations

NA

Summary

This paper introduces a novel method called selective explanations for improving the efficiency and accuracy of feature attribution methods for black-box machine learning models. The key contributions are: - A zero-cost proxy to evaluate the adversarial robustness of deep neural networks without training. - A selective explanation method that detects when amortized explainers generate low-quality explanations and improves them using explanations with initial guess. - An optimization approach for combining amortized and Monte Carlo explanations to improve explanation quality. - Comprehensive evaluation on both tabular and language model datasets, demonstrating improved explanation quality with reduced computational cost. The authors argue that their method addresses the challenges of computational expense in existing feature attribution methods while maintaining or improving explanation quality.

Strengths

- Novel approach: The selective explanation method offers a new paradigm for balancing efficiency and accuracy in feature attribution. - Theoretical foundation: The paper provides rigorous mathematical formulations and proofs for key components. - Comprehensive evaluation: The method is tested on multiple datasets and model types, with comparisons to various baselines. - Practical impact: The approach significantly reduces computational cost while maintaining or improving explanation quality. - Flexibility: The method can be applied to different types of feature attribution techniques and model architectures.

Weaknesses

- Limited exploration of very large models: While the method is tested on language models, it's not clear how well it scales to extremely large models (e.g., GPT-3 scale). - Dependence on amortized explainers: The method's effectiveness relies on the quality of the underlying amortized explainer. - Computational overhead: While more efficient than full Monte Carlo methods, the selective approach still requires additional computation compared to pure amortized methods. - Sensitivity to hyperparameters: The impact of various hyperparameters (e.g., uncertainty thresholds, combination function parameters) is not thoroughly explored.

Questions

- How does the performance of selective explanations scale with extremely large models (e.g., models with billions of parameters)? - Have you explored using more advanced uncertainty estimation techniques, such as those based on Bayesian neural networks? - How sensitive is the method to the choice of amortized explainer? How might it perform with different types of amortized explainers? - Could the selective explanation approach be extended to other types of explanation methods beyond feature attribution (e.g., example-based explanations)? - How might the method be adapted to handle streaming data or online learning scenarios where the underlying model is continuously updating?

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors discuss limitations of their work at the end of Section 6. They acknowledge that the method is currently focused on Shapley values and has been primarily tested on specific types of models and datasets. They also note potential challenges in applying the method to image classifiers. These limitations are reasonably addressed, though a more detailed discussion of potential failure modes or edge cases could have been beneficial.

Summary

This paper proposed a feature attribution method that detects when amortized explainers generate low-quality explanations and improves the explanations with their linear interpolation of themselves and expensive high-quality explanations. To detect the low-quality explanations of the amortized explainers, the proposed method measures the uncertainty of the explanations. Then, based on a selection function defined by the uncertainty, it selects either using the explanations by the amortized explainers or using the linear interpolation of the explanations by the amortized explainers and Monte Carlo explainers. The experiments were conducted on tabular and text datasets, and the experimental results showed that the proposed method achieved better accuracy of explanations than using the amortized explainers and the Monte Carlo explainers solely.

Strengths

- This work is the first to propose a feature attribution method in the "selective" setting, which can improve the quality of the explanations with Monte Carlo explanations when amortized explainers generate low-quality explanations. Since the proposed method is an extension of selective classification and regression, which are successful and well-studied approaches, it is easy to imagine it working well. - The formalization of the proposed method is almost appropriate except that I am concerned about the learned uncertainty (5), and it is described clearly.

Weaknesses

- In common, the quality of explanations has a trade-off between computational efficiency and accuracy. Although the experimental results show that the proposed method can improve the accuracy of explanations, those on computational efficiency are not investigated. - The generated explanations are quantitatively evaluated. However, due to the lack of qualitative evaluation, it is unclear how good the explanations generated by the proposed method are actually from a user perspective. The following are a minor point: - If (5) is MSE, should use $\|| \cdot \||$ instead of $| \cdot |$.

Questions

- **Learned uncertainty:** The loss $\ell(\mathrm{Amor(x;y)}, \mathrm{MC}^n(x,y))$ in (5) is the same as the objective of the amortized explainer $\mathrm{Amor(x;y)}$ in (3). Therefore, if the amortized explainer is fitted enough to the training data, the loss $\ell$ is consistently near zero, resulting in the learned uncertainty function $s$ being consistently near zero, too. This learned uncertainty function does not seem to work well at an inference phase. What is the justification for (5)? - What does 'Random' in Figure 3 mean?

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Yes.

Summary

The paper proposes a method, termed "Selective Explanations," aimed at improving the quality of explanations generated by amortized explainers in machine learning. The authors introduce a technique that detects low-quality explanations and employs a combination of amortized and Monte Carlo methods to enhance them. The approach leverages an "explanations with initial guess" technique, allowing a trade-off between computation speed and explanation quality. The proposed method is validated across different datasets, showing that it can improve the poorest quality explanations typically provided by amortized explainers.

Strengths

1. The concept of using a selective approach to manage computational resources while improving explanation quality is compelling and timely. 2. The proposed method is intuitive and easy to understand. 3. The paper is well written.

Weaknesses

1. The experimental design does not strongly support the major claims on the reduction of computational cost. Evaluations on the tradeoff between explanation quality and computational cost would be helpful. 2. The relationship between uncertainty and explanation quality is unclear, it would be better to have empirical or theoretical proof on their correlations. In addition, the proposed uncertainty measurement also introduces computational overhead when “run the training pipeline for the amortized explainer described in (3) k times” (line 139).

Questions

See weaknesses.

Rating

4

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Given the current state of the submission, the reasons to reject slightly outweigh the reasons to accept.

Response to Author Rebuttal

I appreciate the detailed responses provided by the authors, and my concerns are well addressed. I would like to keep the original accept rating.