Locally Invariant Explanations: Towards Stable and Unidirectional Explanations through Local Invariant Learning

Locally interpretable model agnostic explanations (LIME) method is one of the most popular methods used to explain black-box models at a per example level. Although many variants have been proposed, few provide a simple way to produce high fidelity explanations that are also stable and intuitive. In this work, we provide a novel perspective by proposing a model agnostic local explanation method inspired by the invariant risk minimization (IRM) principle -- originally proposed for (global) out-of-distribution generalization -- to provide such high fidelity explanations that are also stable and unidirectional across nearby examples. Our method is based on a game theoretic formulation where we theoretically show that our approach has a strong tendency to eliminate features where the gradient of the black-box function abruptly changes sign in the locality of the example we want to explain, while in other cases it is more careful and will choose a more conservative (feature) attribution, a behavior which can be highly desirable for recourse. Empirically, we show on tabular, image and text data that the quality of our explanations with neighborhoods formed using random perturbations are much better than LIME and in some cases even comparable to other methods that use realistic neighbors sampled from the data manifold. This is desirable given that learning a manifold to either create realistic neighbors or to project explanations is typically expensive or may even be impossible. Moreover, our algorithm is simple and efficient to train, and can ascertain stable input features for local decisions of a black-box without access to side information such as a (partial) causal graph as has been seen in some recent works.

Paper

References (57)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer aw9L7/10 · confidence 4/52023-06-21

Summary

This work extends [LIME](https://arxiv.org/abs/1602.04938) to generate stable and unidirectional feature attributions through the [invariant risk minimization framework](https://arxiv.org/abs/1907.02893). Their method, LINEX, ensures that it follows the desiderata of faithfulness, robustness to neighborhood sampling, stability (similar to neighbors), and the proposed unidirectionality (closeby examples have the same sign for its attributions). They design LINEX as a concave game (pure Nash equilibrium always exists), thereby ensuring stability and unidirectionality. In the experiments, they show that they can find feature attributions that stay consistent to its neighbors across various modalities (tabular, image, text) and mostly outperform baselines.

Strengths

- The (concave) game theoretic design of the feature attribution method is novel and sound. - The unidirectionality desideratum is original and very sensible. - The theoretical results provide sound support for their design of the proposed feature attribution method. - The proposed method empirically improves upon previous query-based methods. - Stable & robust feature attributions, such as generated by their proposed method, are important for researchers as well as practitioners to gain better insights into black-box models. - The paper is clearly written and easy to follow. - Code is provided.

Weaknesses

- The largest weakness of this work is missing any (novel) qualitative insights into how model work (e.g., similar to Sec. 6.3 and/or 6.4 of [LIME](https://arxiv.org/abs/1602.04938)). The present submission only contains comparisons to previous works and does not show any interesting application of their approach to understand models. - While unidirectionality may be a valid property for most cases, there will be edge cases for which minor differences for close by examples may be discriminative (e.g., a person that has paid back previous loans and another has not, while all other features remain the same). - The trustworthiness of the proposed feature attribution method is not shown. It would be helpful to conduct a user study similar to Sec. 5.4 of [LIME](https://arxiv.org/abs/1602.04938).

Questions

- How is t is set during the experiments? Does it violate Assumption 2? - Are the results for MeLIME correct for CIFAR results since their attributions often just seem to be flipped from LINEX?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

4 excellent

Limitations

The limitations are adequately addressed.

Reviewer bruB6/10 · confidence 3/52023-07-04

Summary

The paper proposes Locally Invariant Explanations (LINEX) as a variation of the well-known LIME explanation method. While LIME is widely used in interpretability, it has been demonstrated to be dependent on the sampling procedure to create neighbouring points as well as the choice of the explanation model. The manuscript aims to mitigate the dependence on the sampling procedure by combining LIME with invariant risk minimisation. The method is evaluated on various datasets, such as IRIS, Fashion MNIST, CIFAR10, Rotten Tomatoes, as well as Medical Expenditure Survey and compared to number of previously proposed neighbourhood selection procedures. Both qualitative and quantitative comparisons using a number of metrics are discussed and it is demonstrated that the proposed method leads to a statistically albeit mild improvement.

Strengths

- The proposed neighbourhood selection mechanism seems reasonable and theoretically grounded. - The quantitative experimental analysis is through. I particularly like that t-tests are used to evaluate statistical significance. - Experiments cover various datasets and modalities.

Weaknesses

- The proposed method modifies the neighbourhood sampling procedure of LIME and is thus rather limited in scope. - The paper is a very hard read. This is not because the covered material is particularly unaccessible but rather a result of a suboptimally structured and at times rather confusing presentation. I will list a few concrete suggestions of how to improve the structure. I want to encourage the authors to thoroughly revise the presentation. This would make the ms a much more accessible and valuable contribution. Concrete suggestions for improving structure: - Explain IRM in more accessible manner (either in the main text or appendix). For example, an environment is currently not properly defined. Furthermore avoid imprecise mathematical definitions such as "the expectation \mathbb{E}_e is defined w.r.t. the distribution over points in the environment" where it is not obvious what is meant by "distribution over points in the environment". - I find the brief introduction to the Nash equilibrium very hard to parse (L 134-145). For example, the utility is defined over joint set actions but in L141 it takes a player k as an input. - Metrics for evaluation, as outlined in Section 4.1, should be moved to the experimental section. Separate paragraphs with corresponding equations and a brief explanation for each of the metrics used in Table 2 should be added. - Assumptions 1 and 2 are hard to parse. Assumption 1 is rather convoluted and should be split up into at least two sentences. Assumption 2 should clearly state that t and \gamma are thresholds and refer to their definitions. - Similarly, refer to the definitions of the environment generation distribution in Definition 2 and explain what is meant by |.| in the indicator functions of equation 2. - The proof sketch L264-277 is very hard to follow and adds little to the overall presentation. Consider to not include it. Minor suggestions for improving writing: - L28 reads as if the doctor is given a cancer diagnosis. I think what is meant is that a patient receives a cancer diagnosis and the doctor has to validate it? - L57 - L63 is rather convoluted. Splitting this up in separate sentences would help. - There are some ill-placed whitespaces such as in L302. - Be consistent with abbreviations, e.g. Supplement vs Suppl. - Avoid a.k.a. in favour of i.e.

Questions

- In L 302, it is stated that SHAP is not a "natural fit". Why is this so? - What was your logic for selecting the particular kernel sizes in L317?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

Limitations are addressed briefly but adequately in the conclusions.

Reviewer Rtbi4/10 · confidence 4/52023-07-05

Summary

Explanation methods are known to be unstable with flickering attributions on inputs with mild perturbations. This work addresses the explanation instability problem by re-posing it as computing attributions that are invariant across different environments (that are obtained through input perturbations). Their proposed scheme exploits Nash Equilibrium guarantees for convex objectives (of LIME) to obtain an explanation scheme with some desirable properties, which they also demonstrate empirically through multi-modal evaluation. Although I found the idea interesting, I did not like the story and have some critical concerns regarding their evaluation. I elaborate them below.

Strengths

- Datasets used for experimental evaluation span various modalities: Image/Text/Tabular. - Connection with NE (Nash Equilibrium) is refreshing and the algorithm is simple. - Theoretical insight: their theorem is intuitive and I like that their estimator regresses all disagreeing attributions to 0, which is clean.

Weaknesses

1. Connection to IRM. I do not see any connection to IRM (Inv. Risk Minimization) setting or method except for the fact that they both use "environments" of some kind. Neither does their estimator borrow techniques from IRMv1 algorithm nor does it impose "invariance" across environments. In that regard, I found their story muddled. 2. Game theoretic perspective. The paper also did not justify well why we should view the estimation problem from game-theoretic standpoint. More specifically, why does the utility function as defined in L230 lead to an explanation method with desired properties. Why should the environments be viewed as agents in a game with competing objectives (and limited resources) at all? 3. I have many evaluation concerns that are described in "questions". 4. The paper is not easy to read because the story is cluttered (first two points), contributions and motivation not straight-forward.

Questions

1. Baselines and ablation. The problem resembles robust mean estimation. I would like to comparisons with the following baselines: 1. S-LIME but with attributions clipped at $[-\gamma, \gamma]$. 2. S-LIME but with attributions aggregated using median instead of mean. 3. (1) + (2) 4. Median-of-Mean estimation instead of mean estimation with S-LIME without increasing the number of function evaluations. 5. (1) + (4) 2. Metrics. 1. Has CAC been used before? I cannot understand well why/how the mean vector matching indicates recourse utility. 2. What happens if we slightly change the defn. of GI to aggregate $|y_b(x') - y_e^{x'}(x')|$? Although similar to INFD, we would then be able to check if the estimator is simply smoothing the explanations while the underlying model is not that smooth. 3. Theoretical analysis and explanation. LINEX performed well on CAC and CI metrics, but I do not understand what is contributing to the improvement. We need ablation study to understand the rrelative contribution of different aspects of LINEX: (a) $\gamma$ clipping, (b) setting non-zero attribution ($w_i$) only when $\sum_e \frac{|w_{ji}|}{w_{ji}}>0$ etc., (c) setting attribution value to the smallest among environments. In other words, a crisp intuitive explanation for why NE leads to empirical gains helps. 4. Figure 3 is hard to follow. LINEX too is highlighting features in the background.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

I do not see them mentioned anywhere.

Reviewer uF3q7/10 · confidence 3/52023-07-07

Summary

In this paper, the authors put forward a local, attribution-based explanation method that fits a local, explainable surrogate (a linear model in this case) to perturbation output of the original model. In essence, this is exact scenario as the immensely popular LIME explanation method; however, this work puts forward LINEX which seeks to improve on the robustness of LIME (where robustness is measured as Fidelity, Stability, Invariance, and a new property put forward as Unidirectionality). LINEX proceeds by learning a collection of linear classifiers, each fitting to a different perturbation “environment” and the authors provide an interpretation of their least-squares fitting approach as a multiplayer game. In addition the authors provide some minor theoretical analysis (Theorem 1) which justifies the modeling choices allowing them to give an intuition for why their method works well.

Strengths

The paper provides clear motivation for the need for its method, provides a straight forward development of the method, and provides salient theoretical and experimental arguments for the method. The experimental evidence in favor of the method is strong, and the method is simple enough that I expect it could be widely adopted into post-hoc explanation tools without too much experimental effort. The paragraph “Implications of Theorem 1” is nice as it allows the reader to very easily digest an interpretation of why Algorithm 1 achieves the goal of the method.

Weaknesses

The method seems to rely on the diversity of the “environments” in a way that is quite unclear to me. Simply picking gaussians with different covariances seems like it will not give rise to the desired benefits of the method (i.e., it is unclear that simply selecting an ensemble of random environments should be enough to improve over using a single superset random environment). I understand that in general we the method picks the lowest magnitude attribution, which can in turn provide greater stability. We observe, indeed, in table 1 that the method provides little benefit in the cases of FNIST, and CIFAR 10, where perhaps these random environments are not enough. However, in IRIS and MEPS, the random environments have significantly better performance than S-LIME, and I am having trouble fully understanding why this is the case. (See questions section). While I understand the authors general point about the desirability of unidirectionality, it seems like it is already generally satisfied in LIME, S-LIME, and MeLIME. The only place LINEX has a considerable advantage in unidirectionality is on IRIS (and to some extent CIFAR). Perhaps to underscore this point, the authors could run experiments on something like Adult or any other financial dataset where their motivating example for using recourse is relevant and see if there is a considerable difference. To be clear, I am not suggesting the authors carry out this experiment during the rebuttal phase, but I am simply stating that having a more clear experimental example demonstrating the importance of unidirectionality would be good in future versions of the paper. There are several works in robustness of explanations (albeit gradient-based and not LIME-based) that are very related to this work but are not included as prior works: [1] Dombrowski et. al. - https://arxiv.org/abs/2012.10425 [2] Wicker et. al. - https://arxiv.org/abs/2212.08507 These are just two of the more recent works that cover robustness of grad. Explanations. I think the more seminal work might be a single, sufficient citation to provide readers with more context: [3] Dombrowski et. al. - https://proceedings.neurips.cc/paper/2019/file/bb836c01cdc9120a9c984c525e4b1a4a-Paper.pdf

Questions

Can the authors further elaborate precisely what they mean by “perform bootstrap sampling to create k different environments.” I would like to understand exactly how is this done step-by-step such that you get such an incredibly meaningful difference between the output of your method and that of S-LIME. Why is there a trailing 1 in Equation (2)?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

See weaknesses.

Reviewer Rtbi2023-08-12

Thanks for the response. Please explain different colors and what is on Y-axis for the plots included in the PDF shared with the general response.

Authorsrebuttal2023-08-12

Figure legend and captions

Thanks for your further response. Legend: - Blue line with circle - LIME - Orange line with inverted triangle - S-LIME - Green line with triangle - LINEX/rand Y-axes labels for each row of the figure: - First row - Infidelity (INFD) - Second row - Generalized Infidelity (GI) - Third row - Coefficient Inconsistency (CI) - Fourth row - Unidirectionality ($\Upsilon$) - Fifth row - Coefficient Attribution Consistency (CAC) We have also mentioned a pointer to the legend in the main figure caption and the Y-axes labels in the sub-figure captions.

Reviewer aw9L2023-08-14

Response to rebuttal

I thank the authors for their detailed response. In particular, I appreciate the discussions on unidirectionality, trustworthiness, and setting t. I think that the mention of the first and last points could be good and interesting additions to the current exposition. Unfortunately, the rebuttal still does not contain any *new* insights beyond the "better than x" style that I specifically asked for in my review. Thus, I’d like to kindly ask the authors again to consider adding such an analysis. This would strengthen the present work considerably in my opinion.

Authorsrebuttal2023-08-15

Example use cases

We truly appreciate the time you are spending engaging with us. We are glad that our response on unidirectionality, LINEX trustworthiness and setting of $t$ were satisfactory to you. Regarding new insights we apologize for somewhat misinterpreting your ask. We would like to point out that we have used LINEX in multiple applications where the end user has preffered it over LIME by a significant margin. We describe two such applications below. If you find these examples to be beneficial we will add (a summarized version of) them to the main paper/supplement. 1. *Explaining misclassified incident tickets:* We worked with experts in the information technology division of a large corporation. This division has an automated incident ticket system that processes customer complaints in text format -- termed as tickets -- for hardware and devices, and tries to identify the problem category. For example, a customer may have memory issues with their devices or network connectivity issues or one of the other hundreds of issues. In a sense, the incident ticket system takes as input a stream of text and tries to predict one of the 300-odd problem classes (eg. memory error, motherboard issue, display issue etc.). In some cases, the system may also run an automated script to mitigate the issue once the problem is identified. Although this is a mature system that serves thousands of clients and has a high accuracy (upwards of 95\%), misclassified tickets are the ones that stand out to the clients. The experts told us that not providing good justification for these misclassifications to the clients severely hampers their trust in the system. We worked with them to test our LINEX algorithm in their framework. The resulting algorithm computes local explanations for classifying IT service ticket texts into the problem classes. What the experts particularly liked about the algorithm was that *it identified mostly the same set of words for similar misclassifications* (for example memory issues misclassified as motherboard issues). This allowed them in some cases to write custom rules to correct for such misclassifications and further improve system performance. This commonality of words we conjecture is because of the unidirectionality and stability properties of our algorithm. LIME did not exhibit similar behavior as it highlighted very different words for similar levels of INFD. Evaluating the algorithmic performance with the experts, we found that the explanations we provided were reasonable in ~79\% of the cases compared with around 40\% with LIME. This evaluation was done on about 100 misclassified tickets handled by the system after deployment. We were thus able to provide better explanations for (roughly) twice the number of cases. The experts said that our algorithm provided the end-users much better intuition about why the system made a mistake (because of commonality of words), showing in most cases that the mistake was acceptable. They felt that this was useful in developing trust in the system. 2. *Financial Fraud Explanation:* We worked with a large financial institution to explain the fraud detection model they had built. The Association of Certified Fraud Examiners (ACFE) claims that roughly 5\% of a companies revenue is lost to fraud every year. Thus, catching fraud or even non-compliance is extremely important for any organization. Their model (fraud $= 1$ else $0$) had $\approx$ 91\% accuracy. The inputs to the model were (transactional) invoices and details corresponding to those invoices such as vendor name, invoice amount, purchase order (PO) or not, vendor address, commodity code, country perception indices (CPI), etc. Since, one of the focuses is to reduce false positives accurate explanations are important. We applied LINEX to this setting to explain why certain invoices were classified as fraudulent. The experts found that in majority of the cases (913 out of 1000) the attributions of LINEX especially in terms of sign made sense. For instance, low CPI implies high risk and so LINEX gave a negative coefficient for this feature for most examples, while LIME gave a positive coefficient for many instances. Going forward their plan is to incorporate such capabilities into their workflow to further improve fraud detection precision.

Reviewer aw9L2023-08-15

Re: Example use cases

I thank the authors very much for their very interesting use cases. I would appreciate it if at least one of them could be included in the main text (in summarized form). Considering the authors' rebuttal and responses to my comments and those of the other reviewers, I will increase my score accordingly.

Authorsrebuttal2023-08-15

Thank you

Thank you for all your comments in improving the ms and reevaluating your assessment. We will add a summarized version of one of the use cases to the main paper and possibly the other to the supplement.

Reviewer Rtbi2023-08-14

Thanks a lot for the detailed response. I now see the connection to IRM through citation [1] better. Although I appreciate the algorithmic similarities between LINEX and iterative algorithm of [1], conceptual connection is still not clear. Invariance is a desired aspect under distribution shifts because of the assumption: _common features have consistent correlation while spurious or specific features have varying correlation across domains_. Which is why invariance can recover common/core features [2, 3]. In order to comment on the applicability of invariance principle to explanation stability, one may need to first understand the cause of variation in explanation across different environments. In my underastanding, varied explanations are because slight variation in inputs may activate very different neuron activation pathways (at least for gradient or perturbation based explanations). With this mental model, it is unclear why invariance or game-theoretic perspective is appropriate for explanation stability. The paper is hard to follow (as also noted by Reviewer bruB) partly because the motivation is unclear. > Even with 2 environments if a feature has same sign attributions it will go towards the lower (absolute value) attribution. While for opposite sign attributions it will tend towards 0. An invariant explanation would set every pixel with varying explanation importance irrespective of their sign to zero. Please correct me if I am wrong. I thank the authors for additional results with S-LIME as requested. It is good to see that none of the simple robust methods for aggrgation of explanations are better than theirs. I also went through the other reviews and agree with the evaluation concerns shared by Reviewer uF3q, aw9L. Instead of simply evaluating the stability or unidirectionality of the explanations across environments, we should also see how LINEX could lead to more faithful/trustworthy explanations. References. 1. Ahuja, Kartik, et al. "Invariant risk minimization games." International Conference on Machine Learning. PMLR, 2020. 2. Piratla, Vihari, Praneeth Netrapalli, and Sunita Sarawagi. "Efficient domain generalization via common-specific low-rank decomposition." International Conference on Machine Learning. PMLR, 2020. 3. Arjovsky, Martin, et al. "Invariant risk minimization." arXiv preprint arXiv:1907.02893 (2019).

Authorsrebuttal2023-08-15

Significant overlap conceptually to IRM

Thank you so much for engaging with us. Below is our response to your concerns. > concerns shared by Reviewer uF3q, aw9L on how LINEX could lead to more faithful/trustworthy explanations Please note that reviewer aw9L seems to be satisfied with our response related to trustworthiness (uF3q hasn't responded yet). They say and we quote "In particular, I appreciate the discussions on unidirectionality, trustworthiness, and setting t. I think that the mention of the first and last points could be good and interesting additions to the current exposition." In particular for trustworthiness we mentioned the following to them: The experiment in Supp. J (Figure 25) demonstrates the trustworthiness of the explanation. Since we claim LINEX to be a better post-hoc explanation compared to the baselines, one of the characteristics we expect is that setting the features deemed important by LINEX to some baseline value must substantially reduce the performance of the model. This is exactly what we show in this experiment using ablation with FMNIST data and comparing LINEX/real to MeLIME. We have now also described to aw9L two use cases in which LINEX was used. > Conceptual connection to IRM There is significant overlap conceptually to IRM. Besides the points we have mentioned before (viz. environments, canceling or suppressing varying features, game theoretic method to solve IRM) we now detail deeper similarities. 1. In IRM, a Structural Causal Model (SCM) is assumed to create the data. For explanations the SCM is just the black box model which creates data (a.k.a. outputs) for the explanation procedures. In fact, since the black box takes an input and outputs some result based on it the SCM can be written as $y=f(x)$ with no latent confounders. The passing of an input through a black box model is analogous to simulating an SCM. Also note that we want to explain just one input at a time which (in all likelihood) has a single ground truth explanation and hence a true explanation model. This is analogous to IRM where in principle there should be a single causal graph (in particular, unique causal parents). Moreover, we are trying to explain inputs in a model agnostic manner and so we do not know if inputs took the same, similar or different paths in the model and in fact, the goal in a certain sense of model agnostic local explainability is to identify that but only through the lens of output behavior, which is why we believe our approach has merit. 2. The environments in IRM have different joint distributions. In our case, we create environments through bootstrapping (other ways also possible) and so the distributions are different in each environment because of covariate shift (as each environment in practice has different $x$'s repeated different number of times). 3. Because of (2) the models trained independently on each environment are thus different for both IRM and our case. > An invariant explanation would set every pixel with varying explanation importance irrespective of their sign to zero. This depends on which aspects one wants an explanation to be invariant to. Since one of the main goals of our paper is to provide unidirectional explanations we want the behavior to be different depending on if the environments produce explanations of opposite or same sign. Opposite sign more strongly indicates that the feature is risky to act on as the output could change in an unpredictable direction. Same sign at least provides some assurance that changing the feature in a certain direction will have the intended direction change in the output. As such, LINEX chooses the lower (absolute) value (amongst the value in the 2 environments) to be as careful and invariant to it as possible. Moreover, from an IRM perspective exact invariance is difficult to obtain in regression and such approximations are used [3]. This is relevant to us since, we are regressing on the class probabilities like LIME.

Authorsrebuttal2023-08-18

Consider re-evaluating your assessment?

We are thankful for all the engagement you have had with us thus far. As you might have seen the other reviewers have mostly been satisfied with our responses and have updated their assessment of our work. Although you might still not be completely convinced about the conceptual connection to IRM (in spite of our last response) we believe we have answered many of your important questions. In particular, i) we provided additional experiments showing other forms of aggregation such as median or median of means does not lead to better results for SLIME. This it seems was satisfactory to you, "I thank the authors for additional results with S-LIME as requested. It is good to see that none of the simple robust methods for aggrgation of explanations are better than theirs". ii) we discussed more deeply the connection to [1], which showed at least algorithmically the connection to IRM of us having a game theoretic approach. This you also acknowledged "I now see the connection to IRM through citation [1] better.". iii) we discussed validity of metrics (viz. GI, CI etc.) and how the $\ell_{\infty}$ constraint implicitly leads to the desired properties. Given these aspects we were wondering if you still believe the assessment should be a "3: Reject"? In our humble opinion this seems a bit harsh. Thank you for your consideration.

Reviewer Rtbi2023-08-21

I thank the authors for their response. I re-read the paper to make sure I did not misinterpret, and went through responses to my questions and other reviews. Here is my re-evaluation of the work. I am somewhat convinced that LINEX is empirically effective as measured using two fidelity related metrics (INFD, GI) and three stability related metrics (CI, $\Upsilon$ CAC). I also acknowldge authors' efforts in providing results for the requested baselines. Despite my best efforts and clarification from the authors, I do not still understand the intuition behind LINEX. Invariant representations is about optimizing representations such that different environments have a shared optimal classifier. LINEX/LIME do not learn any representations, however we may imagine them to be learning a linear projection from the input and a linear regression on top of the linearly projected input, which in effect is just linear regression. We may imagine that input dimensions with varying saliency are filtered in the first step. I could better understand the connection with invariance through this analogy. The connection to invariance, although now is somewhat clear, is still not complete. The connection between core/causal features (in invariant frameowk) and salient features in LIME is not straight-forward. Why is the importance of "truly" salient features expected to be invariant across environments while that of "not-so-truly" salient features is expected to vary? What does variance in feature importance has to with "truly" salient features. Allow me to explain with an example. Say we are explaining a machine learner that classifies a point from a circle, then for classying an example as a point class: the true saliency map should include point as a positive evidence and may also include the circle as a negative evidence. However, if the environments are generated by perturbing pixels, the model may quickly loose sight/salience of the point. LINEX, as I understand, would miss putting salience on the point because it latches to the least importance score acrosss environments. I understand that the setting is somewhat contrived, but goes to demonstrate my confusion that the salience of even "truly" important feature (point) can vary across environments. Perhaps, some latent assumptions need be spelled out. I feel the writing needs a major rewrite to (a) explain intuition behind LINEX or explanation instability, (b) connection with invariance better. Intrdocution and Figure 1 does a good job of pitching the utility of their method without revealing much in to how it works. Section 3 introduces IRM and Nash equilibrium somewhat out of context. Section 4.1 presents desired behaviour of an ideal method and some metrics to measure it. Section 4.2 just mostly presents and explains the steps of their algorithm. The reader is left guessing the intuition behind the design. I congratulate the authors on an empirically successful method, and thank them for all their efforts to resolve my questions. For the above stated reasons, I am retaining my original score. The discussion with authors definitely improved my understanding of their work, but did not convince me to recommend acceptance.

Authorsrebuttal2023-08-21

Thank you for acknowledging our efforts and empirical success, but LINEX will not misclassify the point in your example

> LINEX, as I understand, would miss putting salience on the point... The way the environments are created some examples in each environment (although different number) should have the point while others will not have the point. Based on the classifier output for each example it will be clear that having the point significantly affects the classifier output and this behavior will be true irrespective of the environment. Thus LINEX will pick up on this (point) feature as being an important one for classification. In general, this is the strength of LINEX where it will try to pick persistently important features.

Reviewer Rtbi2023-08-21

Thanks for the response. > ...it will try to pick persistently important features. Yes. But in the example, the point feature although important may have varying importance score across environments because it is a subtle feature when compared to the other feature: not-circle. In other words, why are "truly" important features expected to be also stable?

Authorsrebuttal2023-08-21

Thank you for the continued discussion

As we mentioned above LINEX should still pick up the point feature since its importance will be maintained across environments. Stability is important because if a feature is truly important it should be so irrespective of the neighborhood/environment one picks around the example one wants to explain. LIME is highly sensitive to this and so our method mitigates this issue. Moreover, stability is an emergent property of our method (we don't directly optimize for it) and along with other metrics which check for faithfulness (viz. INFD, GI), our method performing well on all these indicates that it is deciphering the important features more than other methods. Moreover, in Figure 25 in the supplement where we do an ablation analysis by removing features in order of importance we truly see that the features picked by LINEX are more influential than those picked up by MeLIME for instance.

Reviewer uF3q2023-08-14

Thanks for the response

I would like to thank the authors for their detailed response and for the extra effort they put in to run additional experiments. They have addressed a few of my concerns in the rebuttal. My overall opinion is that the paper tackles an interesting problem with solid to strong experimental results. The key drawbacks I see are the lack of clarity in the unidirectionality constraint, and, to no fault of the authors, the difficulty in validating/verifying improved utility of explanations. In light of the authors rebuttal I have increased my score.

Authorsrebuttal2023-08-15

Thank you

Thank you so much for your understanding. If there are any specific concerns you still have about the unidirectionality constraint we would be happy to engage further. Thanks again.

Reviewer bruB2023-08-15

Thanks for the rebuttal.

I want to thank the authors for the detailed rebuttal. I agree that LIME is a widely used explanation method. I do however think that your work has the downside that it only applies to a specific (admittedly important) method in contrast to other methods that aim to robustify a wide range of methods. On the other hand, I think that there is value in proposing a method-specific robustness strategy. Thank you for pointing out that the distribution of environment points is defined in line 121-122. I indeed had missed this. The whole paragraph is quite dense and ideally should be split up. I realize this is challenging due to space constraints. I encourage the authors to change the ms as outlined in the rebuttal as it would significantly improve its readability. I will raise my score to 'weak accept'.

Authorsrebuttal2023-08-15

Thank you

Thank you for revising your assessment and engaging with us. We will update the ms as outlined in the rebuttal. Thanks for suggesting the same.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC