Evaluating the Robustness of Interpretability Methods through Explanation Invariance and Equivariance

Interpretability methods are valuable only if their explanations faithfully describe the explained model. In this work, we consider neural networks whose predictions are invariant under a specific symmetry group. This includes popular architectures, ranging from convolutional to graph neural networks. Any explanation that faithfully explains this type of model needs to be in agreement with this invariance property. We formalize this intuition through the notion of explanation invariance and equivariance by leveraging the formalism from geometric deep learning. Through this rigorous formalism, we derive (1) two metrics to measure the robustness of any interpretability method with respect to the model symmetry group; (2) theoretical robustness guarantees for some popular interpretability methods and (3) a systematic approach to increase the invariance of any interpretability method with respect to a symmetry group. By empirically measuring our metrics for explanations of models associated with various modalities and symmetry groups, we derive a set of 5 guidelines to allow users and developers of interpretability methods to produce robust explanations.

Paper

References (100)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer wVxe5/10 · confidence 4/52023-06-21

Summary

The paper focuses on the robustness of explanations. It begins by defining explanation invariance and equivariance concepts using geometric deep learning formalism and demonstrates that certain popular interpretability methods inherently possess theoretical robustness guarantees. Two metrics, invariance and equivariance scores, are introduced for empirically assessing explanation robustness. These metrics are applied to evaluate various interpretability methods across different modalities. Finally, the paper provides a set of five actionable guidelines to ensure that interpretability methods are employed in a manner that guarantees robustness.

Strengths

1. The paper presents a high-level framework for evaluating the robustness of explanations and introduces two corresponding metrics. Unlike previous work that mostly focuses on saliency-based explanations for image classification, this framework can be applied to various explanation methods (feature-based, concept-based, and example-based) and modalities (images, graphs, and time series); 2. In addition to offering an evaluation framework, the authors also provide guidelines for generating robust explanations. These insights can assist the community in developing improved explanation methods.

Weaknesses

1. The paper's organization could be better aligned with the summaries of contributions provided in the abstract and on page 3. The structure in later sections does not closely follow these summaries, which may make it difficult for readers to follow the narrative. 2. The paper appears to cover many points, potentially leading to the omission of important details. I am interested in understanding how different explanation methods relate to theoretical robustness guarantees (invariant, equivariant), but the paper only presents the results (Table 1) without discussing them. Although the mathematical proofs are available in Appendix D, the main paper lacks an explanation and discussion of these results. For instance, while Table 1 indicates that gradient-based methods have conditional equivariance guarantees, the necessary conditions or assumptions are not explicitly stated. Including explanations or discussions in the main paper would significantly improve clarity. 3. The use of the Dihedral Group for CIFAR10 and STL10 in the experimental section is unclear. It would be helpful if the authors provided an example of the transformations applied to the images in this context.

Questions

1. Could the authors provide further discussion and explanations regarding the results in Table 1, especially for those that are conditionally guaranteed? 2. In the experimental section, what does the Dihedral Group represent for CIFAR10 and STL10? Could the authors share an example of the specific transformations applied to images in this scenario? 3. While there are numerous publications on the explanations for NLP, the paper does not mention the robustness of explanations for NLP. Is it possible to apply the proposed framework to evaluate the robustness of these methods within the NLP domain?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

See weaknesses. In general, I think the paper tackles an important research question in XAI and proposes a valuable framework to address the issue. However, the concerns mentioned in the "weaknesses" section, such as missing or unclear information, may lead to confusion and make it difficult for readers to fully understand the paper. If these related questions can be well answered, I may consider raising my rating.

Reviewer LQbZ6/10 · confidence 4/52023-06-27

Summary

This paper study the robustness of several post-hoc interpretability methods against the transformation of input data. The robustness is measured by invariance and equivariance metrics. Theoretical robustness guarantees and a systematic approach to increase the invariance are derived. Finally, the authors conduct extensive experiments to validate the theoretical analysis of robustness guarantees, using the proposed evaluation metrics. 5 actionable guidelines are derived to improve the robustness.

Strengths

1. The research problem studied in this paper is interesting and novel. Robustness of interpretability against more general input transformation is few studied before. 2. This paper extends the robustness evaluation to other interpretability methods, like example importance and concept-based explanations. These are missing from the current literature. 3. Based on the evaluation, the theoretical robustness guarantees are derived for the popular interpretability methods. 4. The experiment is comprehensive and insightful that 5 practical guidelines are derived for the robustness improvement.

Weaknesses

1. The details about the symmetries are vague and even missing. Which kind of input transformation are considered, rotation, crop, or translation? 2. Since the invariance metric is in conflict with the equivariance metric, it may be confusing for users to decide which metric is more suitable, given the symmetries. For example, if the small perturbations are added into the input data, the invariance of feature explanation is expected. 3. The simple Monte Carlo Sampling is inefficient for the evaluation of two robustness metrics, especially for the rare events. 4. The ImageNet dataset should be considered for the experiments.

Questions

1. Please provide more details of the symmetry group. How does the change of symmetry group affect the robustness evaluation results? 2. What's challenges when applying proposed evaluation method to real world dataset and interpreter?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Please refer to the weakness

Reviewer u2n85/10 · confidence 3/52023-07-02

Summary

This paper proposes the definition of the robustness of explanations with respect to the model symmetry group. For models invariant to some symmetry group, the explanation should also be invariant or equivariant to it. The paper derives two metrics to measure the invariance and equivalence of explanations and analyzes the robustness requirement for three explanation methods (e.g., feature importance needs to be equivariant, and example importance and concept-based explanations need to be invariant). It also theoretically analyzes the guarantees of different methods in robustness. In the end, the paper proposes to improve the robustness by aggregating the explanations over several symmetries. Experiments show that different explanation methods have different invariance/equivariance properties.

Strengths

1. The paper is very well-written and easy to follow. 2. The relevant literature is summarized comprehensively. The considered problem, evaluating the robustness of explanation from a geometry perspective, is not well-studied before. 3. The concepts of explanation invariance and equivaraince are novel. The evaluation metrics are sound. 4. Multiple explanation methods are evaluated in the experiments.

Weaknesses

1. The biggest weakness is the limited scenarios that can use the proposed evaluation methods. The paper only considers models that are perfectly invariant. Please see Limitations for details. 2. The requirement of the model's invariance is not well defined. Under some group transformations, while the hard-label prediction of the model (i.e. after argmax) is unchanged, the soft-label prediction (i.e. after softmax but before argmax) might be changed. The model is invariant from the first perspective. However, from the second perspective, the model is non-invariant and the explanation should also be non-invariant/non-equivariant. I guess that is why the saliency map is not equivariant while the model is considered invariant in Appendix I. 3. The symmetry groups considered in experiments are limited.

Questions

1. The paper only considers the post-hoc interpretability. Is it possible to extend the concept of explanation invariance/equivariance to other methods? For example, the attention should be equivariant under input translation? 3. When the explanation space is not identical to the input space, what is group representation $\rho'$? 4. Is there any trade-off between the robustness and the utility of the explanation? Minor: 1. The colors in Figure 2 are hard to distinguish. 2. What is "Dihedral Group" in Table 2?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

4 excellent

Contribution

2 fair

Limitations

The proposed definitions and evaluations heavily rely on the assumption that the model is perfectly invariant. I appreciate the experiments and discussions in sec 3.3 which suggest that there's no linear relationship between model invariance and explanation invariance/equivariance. Therefore, models not designed but trained for invariance can not use the proposed methods. However, this assumption limits the model architectures and group transformations considered and thus limits the application of the proposed methods.

Reviewer NZiw5/10 · confidence 3/52023-07-07

Summary

The core contribution of this work is the direction of measuring explanation robustness through a more broader set of data perturbations/transformations for different data modalities which have not been discussed in literature thus far. For instance shift transformations in images, cyclic translations in time series, etc. The authors define concepts of explanation invariance and equivariance and expose their theoretical properties. They conduct experiments to measure these types of invariances for 3 different types of explanations and several datasets and also present guidelines on which invariances to measure in which use cases.

Strengths

Overall, making explanations more trustworthy is an important area to investigate. Measuring robustness of explainability methods is valuable for their use in practical deployed systems. - Authors define several types of perturbations for different data modalities. Authors define the concepts of invariance and equivariance for different data modalities and show their theoretical properties. - Authors have presented a broad collection of experiments with several data modalities and explainability types to show that many existing explainability methods are not robust as measured via invariance & equivariance.

Weaknesses

There seem to be links between model robustness and explanation robustness which are not explored in the draft. For example, if the model is not robust to certain types of suggested data perturbations (when we might expect it to), then would it not make sense to measure if the explanations are not in fact the same? Because the same top reason/explanation why a data sample was classified in class A cannot also be the reason why its not classified in class A.

Questions

- Is there any reason why text models were not evaluated? It could make sense in tasks for e.g. summarization, etc. if certain facts are shifted from the beginning to the end of a paragraph and if the importance score on words still come up right. - In your experiments are you assuming that the models you pick are already robust to the type of perturbations you add to the data to measure explanation robustness(invariance/equivarance)? Or is explanation invariance being measured irrespective of whether the model is robust or not.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

See above.

Reviewer 7REw6/10 · confidence 4/52023-07-07

Summary

In this paper, the authors propose a set of desiderata for explanation methods for neural networks ranging from CNNs to GNNs. They postulate that any explanation that is able to faithfully explain the model should be in agreement with the invariance properties exhibited by the underlying model. They formalize this idea by introducing explanation invariance and equivariance with reference to specific symmetry groups. Using this formulation, they derive metrics that measure robustness of several interpretability methods and some theoretical guarantees. Their experiments verify explanation robustness for models trained on diverse modalities such as images, time series and tabular data. They provide guidelines for developers to develop robust model explanations using their metrics.

Strengths

Evaluating explanation approaches is an important area of research not only for evaluating existing approaches but also to aid practitioners in developing new explanations that are robust. The paper is written very clearly albeit a bit dense to parse – this does not take away from the general reader’s experience of the paper, but it could use a more exemplar way of introducing concepts (Specific comments below) The explanation methods and the models tested encompass several modalities. This is a real strength of the paper as explanation evaluations are usually limited to analysis of salience maps.

Weaknesses

1. Dense nature of the writing: While I appreciate the authors’ efforts in introducing the concepts of geometric priors and group symmetry to the readers, the paper could be more clear in explaining what transformations are being considered in different symmetry groups. For instance, what transformations are contained in the D8 symmetry group for the CNN based examples? A table to this effect can help the readers understand what transformations are being evaluated. 2. Example results (positive and negative): The discussion of the results is rather pedantic, examples of why improving invariance and equivariance would make sense given an invariant model would help the reader understand the importance of the metrics. Without this context, it is hard to differentiate, say, an invariance score of 0.9 to a 0.5. 3. Why only exact invariance and equivariance? I wonder if in its current form the metrics are too rigid. As the authors mention in the paper, most networks for real world problems are only approximately invariant or equivariant. In fact, characterizing such property is a hard problem in itself. In these cases, it’s hard to see how the current approach can help. 4. This approach is limited in that the only transformations that are considered are geometric in nature. While this is alluded to in the appendix, more real world examples of invariance are from non-trivial corruptions such as measurement error or signal degradation. 5. The paper presents no conclusion and more discussion + visualization of the results is needed, especially in cases where the metrics fail to capture the expected behavior.

Questions

1. Can the metrics be extended to approximately-invariant/equivariant networks? What are the pros and cons of such an extension? 2. Are invariance and equivariance the only properties that need to be considered for explanations from a geometric perspective?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Please refer to weaknesses above. There is not much potential for a negative societal impact - evaluating explanations would only serve to impact society positively.

Reviewer wVxe2023-08-12

Thanks for the authors' response. I have decided to maintain my initial rating of the paper. My primary reservation remains, "The paper appears to cover many points, leading to the omission of important details.", a concern echoed by both Reviewer 7REw and Reviewer LQbZ. The absence of these key details makes the paper difficult for readers to fully grasp, even though some further details are provided in the appendices. I would recommend the authors reconsider the paper's structure in the next version, whether for NeurIPS or elsewhere.

Authorsrebuttal2023-08-14

We thank the reviewer for their response. While we respect the reviewer's decision, we note that the reviewer was willing to update their rating if *related questions can be well answered* in their original review. We believe that our rebuttal provides thorough answers to the reviewer's point about omissions, all of which have been used to update the manuscript. This includes: * A detailed explanation on the theoretical results from the appendices in Section *Theoretical robustness guarantees* of our rebuttal. This discussion has been added to *Section 2.2* of the manuscript. * A clarification on the symmetries used in our experiment in Section *Clear explanations for symmetry groups*, hence addressing the concerns both Reviewer 7REw and Reviewer LQbZ had about clarity. These details have been added to *Section 3* of the manuscript. * An extension of our formalism to NLP applications, verified theoretically and empirically, in Section *NLP applications* of our rebuttal. These additional results have been added to a new appendix. While all of these changes have been integrated to the manuscript, it is unfortunately impossible for us to upload the updated version on OpenReview. For this reason, we would like to ask the reviewer some additional questions. Was the reviewers satisfied by the answers we provided in our rebuttal? If that is the case, what changed the reviewer's mind on updating their rating? If that is not the case, which specific points should we clarify? We would be very happy to take advantage of the discussion phase to address any residual concern the reviewer might have.

Reviewer LQbZ2023-08-19

Response to Rebuttal

Thanks for the rebuttal. The authors' responses address most of my concerns.

Authorsrebuttal2023-08-19

We thank the reviewer for their feedback. We are delighted that our rebuttal addressed the reviewer's concerns.

Authorsrebuttal2023-08-20

Rebuttal Summary

We wish to thank again all five reviewers for their constructive feedback which has helped us to improve the clarity and contribution of our work. We are delighted that all five reviewers recommend acceptance on our paper. **With the fast-approaching end of the discussion period, we wanted to briefly summarize our responses to the 3 reviewers who have not yet responded to our rebuttal**. ## **Reviewer 7REw** | **Their Request** | **Our Rebuttal** | |---|---| | Clarifications on the symmetry groups used in our experiments. | A detailed table explaining what each symmetry group represent and how it acts on data. | | More context to the raw values of the robustness metrics. | Qualitative analysis of various examples of images for which the explanation is not robust, with varying degrees of robustness violation. | | Possible extensions to explanations of models that are not perfectly invariant. | Re-emphasize the discussion from Section 3.3 of the paper, which shows precisely that. We also provided an extension of our formalism to measure the robustness of explanations of invariant language models. | | A conclusion to the paper. | A two paragraph conclusion that has been appended to the manuscript. | ## **Reviewer NZiw** | **Their Request** | **Our Rebuttal** | |---|---| | Possible extensions to explanations of models that are not invariant. | Re-emphasize the discussion from Section 3.3 of the paper, which shows precisely that.| | Possible extensions to explanations of language models. | A theoretical and empirical study of our robustness metrics for explanations of bag-of-words models.| | Clarifications on the robustness assumptions made in our experiments. | A detailed description on the invariance assumptions made for all models used in our experiments. | ## **Reviewer u2n8** | **Their Request / Criticism** | **Our Rebuttal** | |---|---| | Limitations to models that are perfectly invariant. | Re-emphasize the discussion from Section 3.3 of the paper, which shows precisely that we can also record the invariance/equivariance of explanations when the invariance of the model is slightly relaxed. We also provided a theoretical and empirical study of our robustness metrics for explanations bag-of-words models to re-emphasize the wide applicability of our framework across various modalities (on top of the 4 modalities already presented in the manuscript).| | Clarifications on the robustness assumptions made in our experiments. | A detailed description on the invariance assumptions made for all models used in our experiments. | | Possible extensions beyond post-hoc interpretability. | Some details on how our robustness metrics could be used to assess the explanations outputted by interpretable models (with the example of attention attribution).| | Some explanations on how to choose a representation for the symmetry group when the input and explanation space don't match. | A detailed illustration on how to make this choice with the example of mask feature attribution methods. | We hope that these 3 reviewers will respond before the (fast-approaching) end of the discussion period and be satisfied that we have addressed their concerns and collectively improved the paper on the basis of their feedback. Otherwise, we hope that the AC will account for our rebuttal to the assessment from these reviewers.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC