When users receive either a positive or negative outcome from an automated system, Explainable AI (XAI) has almost exclusively focused on how to mutate negative outcomes into positive ones by crossing a decision boundary using counterfactuals (e.g., \textit{"If you earn 2k more, we will accept your loan application"}). Here, we instead focus on \textit{positive} outcomes, and take the novel step of using XAI to optimise them (e.g., \textit{"Even if you wish to half your down-payment, we will still accept your loan application"}). Explanations such as these that employ "even if..." reasoning, and do not cross a decision boundary, are known as semifactuals. To instantiate semifactuals in this context, we introduce the concept of \textit{Gain} (i.e., how much a user stands to benefit from the explanation), and consider the first causal formalisation of semifactuals. Tests on benchmark datasets show our algorithms are better at maximising gain compared to prior work, and that causality is important in the process. Most importantly however, a user study supports our main hypothesis by showing people find semifactual explanations more useful than counterfactuals when they receive the positive outcome of a loan acceptance.
Paper
Similar papers
Peer review
Summary
This paper proposes a framework for "semi-factual" explanations - explanations given to an applicant who receives an accept decision, about what they could do which would allow them to maintain that decision. The framework and objective propose utilize a causal model of the action space, and consider the applicant's gain in utility, the plausibility of the output, the diversity of suggestions and the robustness of the suggested actions. Experimentally, they demonstrate an improvement of their approach over baselines and also conduct a human study showing their approach is perceived to produce more useful explanations.
Strengths
- incorporation of causal models in counterfactual-type explanations is a good direction and a sensible idea - the presented framework is quite thorough in terms of the desiderata you'd hope for in semi-factual explanations - the human study at the end is very cool - I haven't seen so many of these in this literature and I think that really sets this paper apart
Weaknesses
- Overall, I find myself lacking quite a bit of clarity around many of the formalisms in pages 3-6 as the framework is described. Some of this points of confusion are: 1. I find 3.1 hard to follow - for instance, it's not clear to me what kind of object A is: it seems to be from a subset of the domain of X, but then it's stated later that they're not simply added, so I'm not sure how to interpret its value, or what it means that "actions have to be mutable" (line 94); I'm not so sure why we need S if we have S_M, what is S supposed to be? 2. It's not obvious to me what P is supposed to be - should it be conditional on x rather than a marginal? is it defined with reference to some action history? How do we define it in a way which serves us? Also, I'm not sure why P and G should be multiplied - may need more explanation for that - I found Section 4 challenging to follow as well: 1. it's not clear to me what a "causal" vs a "non-causal" domain is supposed to be 2. I didn't follow the methods for empirical approximations of H or M, and I'm not sure why P needs an approximation if it's just calculating a known density that we define? 3. I don't really follow the fitness function section either but I'm not so familiar with genetic algorithms so that could be why - I'm not sure that Lemma 3.3 isn't trivial - if H >= 0 for all a, and we assume G > 0 and P, R, \lambda are > 0 by definition, doesn't J have to be >= 0 by definition as well? Smaller notes: - how dow we know S inverse exists? (line 105) - line 120: a little confusing referring to post robustness before it is defined - line 216: depict is an odd word here, present may be better - how are the results in Fig 2 normalized? It's not clear to me that they should be - why is P lowers for S-GEN? Is this good or bad?
Questions
- Is there work from the counterfactual (rather than semi factual) literature that uses causal modelling? I'm not sure how deep the novelty is supposed to run here
Rating
3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
2 fair
Presentation
2 fair
Contribution
3 good
Limitations
I recommend against using COMPAS as a dataset in papers which are not in a criminal justice context - see "It's COMPASlicated: The Messy Relationship between RAI Datasets and Algorithmic Fairness Benchmarks" for more, and it's probably worth writing a piece about this.
Summary
This paper looks into the mirror problem of counterfactual recourse. In the latter the goal is to determine what is needed to change a negative outcome to a positive such as in loans. In the former the goal is to determine to what extend a positive decision would remain and to what extend the end user could have pushed their features. The paper introduces this novel view with motivations from psychology and also investigates this with a user study. In order to achieve this they use similar ideas from causal algorithm recourse with constraints on the proposed explanations.
Strengths
- The paper tackles a new problem that has not had much attention in the XAI community. We are often focused on the algorithm recourse, however, this paper proposes a new twist to explanations when users in fact got a positive outcome. The user study albeit only 15 samples (might be wrong on this) shows that people do in fact prefer this type of explanation when in fact getting a positive outcome. Which tbh makes sense as in why would they care about what would have happened if they didn't get it? - Another plus is the simplicity of the idea which i like very much given that it is well-motivated. - The paper also clearly introduces their desiderata for the explanations as well as the objective they aim to solve. - Lastly, just like in the previous causal algorithm recourse, they also provide a causal view of this problem.
Weaknesses
- The paper seems to take too much inspiration from the causal algorithm recourse paper to a point where it becomes just a rewriting of that paper from a different view with the exact same/similar techniques. - I also don't understand how gain and cost are not equivalent as you can have causal costs as well. in your line 145 you say the main difference is the causal part which I don't get as this can also be done with cost no? - In Figure 2, what happened to plausibility? does that mean the explanations are all non-plausible? -
Questions
see above
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
2 fair
Limitations
NA
Summary
The paper formalizes the notion of semi-factual for positive outcomes. Given a black-box classifier, a semi-factual is a statement which informs a user about “even if” scenarios in which they would maintain the assigned positive prediction, increasing their “gain”. The authors formalize the problem for both a causal and non-causal setting. They show the validity of their method, S-GEN, via experiments on real-world datasets. Lastly, they also performed a first user study which shows the usefulness of semi-factual for users.
Strengths
To the best of my knowledge, the topic is novel and the proposed ideas are interesting for the algorithmic recourse field. The paper is clear and it can be understood quite well. The related works are discussed extensively and the problem is nicely framed in the context of the state-of-the-art literature. I liked the causal formulation of the semi-factual problem and the user evaluation. More specifically, the user study provides strong hints that semi-factuals have merits, thus it grounds the algorithmic recourse research in the real world.
Weaknesses
The notation and the various proofs can be hard to read at first glance. For example, in Section 3.1., between lines 95 and 112, having $S$ to be the counterfactual and $\textbf{S}$ to be the structural equation instead can cause confusion.
Questions
- If we consider the notion of Gain, how does it relate also to the concept of Shapley value [1]? Which are the differences/similarities between them? - In Section 3,.4, on line 181, how do you define such distribution density? Namely, given the SCM, you can define the interventional joint distribution after a do action $Pr(\mathbf{X} | do(\mathbf{a}))$. Is it the same thing? - In Section 4, how do you define the “empirical approximation” of $G$, $P$, $H$, $R$ and $J$? How can you learn such approximations? - In Section 6, on line 333, the paper says “...rate on a scale from 1-5 how useful each where.”. Could you elaborate more on this point? Useful to what? [1] Lundberg, Scott M., and Su-In Lee. "A unified approach to interpreting model predictions." Advances in neural information processing systems 30 (2017).
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
The authors adequately address all the limitations.
Summary
This paper explores techniques for optimizing positive outcomes using semi-factuals. To do so, they propose the concept of "gain" and set up a small user test to demonstrate the usefulness of their technique.
Strengths
The authors offer an interesting and compelling framework for optimizing positive outcomes, with strong formal notation and proofs.
Weaknesses
My biggest concern with the paper is its presentation. I think the authors still have much work to do in cleaning up the writing and presentation of this work. See below.
Questions
-- I think most importantly, the authors need to better motivate their work, especially in the Introduction. I think it might help to give more context of previous work in the Introduction: in what contexts is this kind of work used? what's come before? how does this work add on to it? The answers to these questions are scattered throughout the rest of the paper, and in Related Work, but it should be very crisply and tightly explained in the Introduction. In general, the language in the Introduction is a little too loose. I'd love a very clear distinction between counterfactuals and semi-factuals, and between positive outcomes and negative outcomes. I see that loans are one context in which this work could be useful, but what are other contexts in which this work could be useful? This is just a smattering of ideas for cleaning up the Introduction, but I think in general, the Introduction feels a little too philosophical. At some point in the Introduction, I'd like it to be made clear how this is used in the context of ML. -- There are also some strong claims made that I would want justified (e.g. "completely unexplored" in line 22). -- I wasn't convinced by the two main issues outlined in the second paragraph of the Introduction. These are the two main issues according to who? Fairness is mentioned in line 37, but it wasn't clear how fairness and ethics is really an issue in this paragraph. (I understand what the authors are trying to say, but the writing doesn't make it as clear as it could be.) -- There are typos scattered throughout the manuscript, which should be proof-read carefully. -- It might help to move part of lines 61-65 into the Introduction. Also, I think describing the "urgent need" in line 57 would help motivate this work, too.
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
1 poor
Contribution
3 good
Limitations
The authors do mention the limitations of their user evaluation in the Discussion, but I feel they could expand on the limitations of their approach a bit further.
I thank the authors for the extensive rebuttal and for clearing my doubts! I do not have additional questions since I think the suggestions and improvements the authors made (also for the other reviewers) resolved most of the issues.
Response
Thanks for your rebuttal - I'll respond point by point here: Novelty: to clarify, is the novelty here in the optimization framework for semi-factuals? How does this differ from the image-based works in the beginning of Sec 2 - do they not contain optimization frameworks as well? Object A: I'm still not so clear - in general I understand we might want some more complex operation to combine A with X to produce a new point, but I don't see then how A can be from the domain of X. Additionally, I don't understand how "race" could be a potential action, isn't that a feature? (mutability aside) S and S_M: I'm still not sure I understand - they both seem to be from X* x A* -> X*, defining semi-factual interactions. Don't they both define the same function - if S_M is associated with an SCM, wouldn't it all be associated with S? P: what is a plausibility function? It seems like there's quite a bit around P which needs to be added to the paper for clarity's sake. P conditional on X: I meant, should it be conditional on the initial value of X (rather than the full dataset as you've defined it) Causal vs non-causal: I guess this is nit-picking but I would find it clearer if you referred to these as two "modelling approaches" rather than "domains" - whether or not you happen to know an SCM is not a feature of the domain In general, it's hard for me to re-evaluate clarity without seeing the draft. It seems that at the moment we still have a few major points where I'm not totally following - for this reason I'm inclined to not change my score. However, I see that I'm the odd one out here among reviewers so it's possible that I'm missing background within this literature, so I will consider this as I discuss with them.
Author(s) Second Response (Part 1)
We appreciate the follow-up questions and do our best to clarify all residual concerns. Note to respond adequately, we have divided this response into two parts (this is the first). *** *Novelty: to clarify, is the novelty here in the optimization framework for semi-factuals? How does this differ from the image-based works in the beginning of Sec 2 - do they not contain optimization frameworks as well?* Correct, our novelty is primarily with our optimization framework for semi-factuals. However, we do also consider our user study a major contribution (see lines 40-41), as e.g. [3] recently discussed “the paucity of user studies” as a major pitfall in current semi-factual research (indeed the reviewer also graciously noted this sets our paper apart). Yes, the image-based works in Section 2 do contain semi-factual optimization frameworks also, however, to answer your question ours has two important differences: Firstly, prior work typically defines a “good” semi-factual just by how large the $L_2$ distance is between the test instance and the generated semi-factual explanation [2]. In contrast, we define good semi-factuals as those with high “Gain” scores, which is designed to be more meaningful in application by purposefully aiming to change features in a way that would benefit a user and correspond with “even if” thinking (see lines 126-137). Secondly, our framework “flips” the usual recourse setup to focus on *optimizing positive outcomes*, rather than *changing negative outcomes into positive ones*, to the best of our knowledge, this is a novel problem formulation for explainable AI. *** *Object A: I'm still not so clear - in general I understand we might want some more complex operation to combine A with X to produce a new point, but I don't see then how A can be from the domain of X. Additionally, I don't understand how "race" could be a potential action, isn't that a feature? (mutability aside)* A is not from the domain of X but from an action space parameterized with X (defined in line 90-91). It just means that the action space for each feature depends on the feature itself. For example, regarding a feature “employment”, the action space could be {become employed, become unemployed, become self-employed}, which depends on the characteristics of the specific feature, this is all we mean. Sorry there appears to have been confusion partly because of the space constraints prior. You are correct that “Race” is a feature and not an action. To be clearer and expand our original rebuttal we can say: *By “actions have to be mutable” we mean that the feature should be possible for the user to change, e.g. not “doing the action” of changing a “Race” feature from Race A => Race B…”* *** *S and S_M: I'm still not sure I understand - they both seem to be from X * x A * -> X * , defining semi-factual interactions. Don't they both define the same function - if S_M is associated with an SCM, wouldn't it all be associated with S?* No, they do not define the same function. S_M associates with a causal model M, whilst S does not consider the causal model. That is, with S, the only changed features correspond to the actions of x. However, in S_M, other features with dependence on the action-changed features might also change accordingly. For example, a user might decide to do the action of lowering their income, in S this is the only feature that would change, but in S_M, it would also naturally change the “dept to income ratio” feature due to the causal dependencies.
Author(s) Second Response (Part 2)
We appreciate the follow-up questions and do our best to clarify all residual concerns. Note to respond adequately, we have divided this response into two parts (this is the second). *** *P: what is a plausibility function? It seems like there's quite a bit around P which needs to be added to the paper for clarity's sake.* A plausibility function as we define it here is simply a function which outputs a value corresponding to how “plausible” a semi-factual explanation is. The plausibility of a generated explanation is extremely well studied in the literature (e.g., see [1,2,4] for just some examples). In our case we simply use the distance of generated semi-factuals to known training data to do this here in non-causal settings (see lines 254-259; similar to [2]). But again, plausibility is naturally taken care of (at least how we define it here) thanks to the SCM in causal settings. For what it’s worth, we don’t feel this will be difficult to articulate further in the paper to be crystal clear. Indeed, one simple sentence at the end of our “Plausible Gain” paragraph on page 5 should do it. We would have included this in our initial response to the reviewer but had no space left, so we can thankfully do it here: **Revision:** We add this at the end of the “Plausible Gain” paragraph on page 5: ” *“Note however, in our non-causal tests we use the $L_2$ distance to the actual training data to approximate plausibility (i.e., being in distribution, similar to [1,2,4]), but this issue of plausibility is naturally taken care of in our causal tests thanks to the SCM ensuring plausible feature mutations, so we don’t explicitly consider plausibility there going forward.* *** *P conditional on X: I meant, should it be conditional on the initial value of X (rather than the full dataset as you've defined it)* Thanks for clarifying this, and sorry for the misunderstanding. Indeed P could be conditional on X only. However, in practice, it is hard to get the true P, and we thus appealed to an empirical approximation (line 257) as most work in this area does (see [1,2,4]). In our setting, the empirical approximation is built using the training dataset directly. *** *Causal vs non-causal: I guess this is nit-picking but I would find it clearer if you referred to these as two "modelling approaches" rather than "domains" - whether or not you happen to know an SCM is not a feature of the domain* No problem, done. *** *In general, it's hard for me to re-evaluate clarity without seeing the draft. It seems that at the moment we still have a few major points where I'm not totally following - for this reason I'm inclined to not change my score. However, I see that I'm the odd one out here among reviewers so it's possible that I'm missing background within this literature, so I will consider this as I discuss with them.* We hope this has clarified the points above which remained unclear, if not please let us know and we are happy to do more, thanks again. *** [1] Laugel, Thibault, et al. "The dangers of post-hoc interpretability: unjustified counterfactual explanations." IJCAI. 2019. [2] Kenny, Eoin M., and Mark T. Keane. "On generating plausible counterfactual and semi-factual explanations for deep learning." AAAI [3] Aryal, Saugat, and Mark T. Keane. "Even if explanations: Prior work, desiderata & benchmarks for semi-factual XAI." IJCAI – 2023 [4] Van Looveren, Arnaud, and Janis Klaise. "Interpretable Counterfactual Explanations Guided by Prototypes."
I thank the authors for their reply! I appreciate the updates. I think it's hard to evaluate the clarity of the new draft without being able to see it, but assuming the clarifications are adequate, I'll change my score to a 6.
Author(s) Followup
We thank the reviewer for their followup and engagement with our work! We noticed the reviewer didn’t actually update their score, and we are just wondering if this was a simple oversight, or if you prefer to see larger portions of the new draft to do so? We note the primary areas of concern are simply (1) the Lit. Review paragraph about user studies, (2) the limitations, and (3) the intro. If so, we are happy to present the relevant revised paragraphs here in a comment, please just let us know.
Thanks for the clarification
I have read the reply and would like to thank the reviewers for the clarifications. I have an additional question however, you state that your main contribution is not the optimization but rather the introduction of the concept. Hence, could the authors please tell me exactly what the differences to are "On generating plausible counterfactual and semi-factual explanations for deep learning." besides that they are applying it to images. Please be specific and clearly state your novelty besides the new data type setting. I would also like to note that the paper has been cited 74 times and hence urge the authors to be specific on the related works. I will myself look through the citations and cross check your answer later. Thanks
Author(s) response to additional question
We thank the reviewer for their engagement with our work. We start by first addressing how our research differs from Kenny & Keane [1], and then the 74 citations. ### Difference to [1] Our novel “concept” is to optimise positive outcomes, in [1] and all 74 citations, no one else has considered this. This is important because despite their growing popularity, it is not known how semi-factuals (SFs) could be useful for ML. Indeed [5] discuss this issue explicitly *“it is… unclear… how these types of explanations are useful…”* [5]. So, to rectify this, we showed in our user study that optimising positive outcomes with SFs is useful. To actualise this, we made many advances to [1]. | | Kenny & Keane [1] | Ours | |--------------------|-------------------|----------------------------------------------------------------------------------| | Framework| Modify features of low probability in the counterfactual (CF) class to be their expected values. Stop this one step before the decision boundary to generate SF. | Maximise the “distance” between a test instance and its SF using an SF specific optimisation with a gain function, actionability constraints, and SCMs. | | Theoretical Contribution | None| Yes, verification of the SF objective and comparison to CFs.| | User Study| None| Yes, showed a use case in which SFs are more useful compared to CFs (something missing from the XAI lit. but desired [5,7]) | | Causality Considered |No| Yes, full derivations, objectives, theory, and experimental results.| | Categorial features considered| No|Yes| | Problem Setting| Deep learning – image classification | Algorithmic Recourse| Note the distinction between images and tabular is important in recourse [2], and we outperformed Kenny & Keane [1] significantly in comparative tests. ### 74 Citations Of these, 5 propose a new SF algorithm, 4 are not directly related to XAI, 1 uses SFs for evaluation, 9 are surveys, 29 cite in relation to CFs, 4 are related to CBR, 4 discuss user testing (but don’t do it), 8 discuss SFs, 1 is from the psychology, and 9 we classify as “Other” (e.g. a thesis). We now detail the 5 SF algorithms (we didn’t cite [3,4] in our paper since they are not published): [3] use a joint gaussian mixture model to sample CF (and SF) images in a deep learner. They don’t consider individual features, categorical features, recourse, users, causality, gain, or positive outcomes. (not published) [4] was released on arXiv the same month as the NeurIPS deadline, so as per NeurIPS guidelines we do not consider this. (not published) [6] and [8] both use standard CF approaches with generative models on images similar to [1], and stop before the decision boundary to make SFs. Both (and [1,3]) see SFs as an "afterthought" to CFs, ignore users, categorical features, theory, recourse, and positive outcomes. The most relevant work is [5], as they propose an SF specific algorithm. They suggested using diverse SFs to explain reject decisions. The authors note that their method (1) likely produces implausible explanations, and (2) that they are not clearly useful, partly because how to formalise SFs is unclear. Here, we address this: (1) We guarantee plausibility with SCMs and actionability constraints, and (2) we showed people find our problem formulation and explanations useful with our user study. ### Conclusion We trust the reviewer can see how our work differs and advances SF research. Above all, our concept of using SFs to optimise positive outcomes has compelling evidence (in our user testing) that it is **perhaps the first useful formulation for SFs in ML**, which is a big step forward for SF research So, to be clear, aside from the framework of optimising positive outcomes, our Gain functions, theory, and user study are completely novel compared to all prior work. Thanks P.S. We post all cites and our categories of them below so the reviewer may more easily fact check us. *** [1] Kenny, Eoin M., and Mark T. Keane. "On generating plausible counterfactual and semi-factual explanations for deep learning." AAAI [2] Verma, Sahil, et al. "Counterfactual explanations and algorithmic recourses for machine learning: A review." arXiv [3] Xie, Zhouyang, and Duanbing Chen. "Joint Gaussian Mixture Model for Versatile Deep Visual Model Explanation." [4] Dandl, Susanne, et al. "Interpretable Regional Descriptors: Hyperbox-Based Local Explanations." arXiv [5] Artelt, André, and Barbara Hammer. "“Even if…”–Diverse Semifactual Explanations of Reject." IEEE. [6] Vats, Anuja, et al. "This changes to that: Combining causal and non-causal explanations to generate disease progression in capsule endoscopy." ICASSP [7] Aryal, Saugat, and Mark T. Keane. "Even if explanations: Prior work, desiderata & benchmarks for semi-factual XAI." IJCAI [8] Zhao, Ziwei, et al. "Generating Counterfactual Images: Towards a C2C-VAE Approach."
Our Categorisations of Kenny & Keane's Citations (part 1)
## Other SF Algorithms Xie, Zhouyang, and Duanbing Chen. "Joint Gaussian Mixture Model for Versatile Deep Visual Model Explanation." (2022). Dandl, Susanne, et al. "Interpretable Regional Descriptors: Hyperbox-Based Local Explanations." arXiv preprint arXiv:2305.02780 (2023). [5] Artelt, André, and Barbara Hammer. "“Even if…”–Diverse Semifactual Explanations of Reject." 2022 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, 2022. Artelt, André, and Barbara Hammer. "“Even if…”–Diverse Semifactual Explanations of Reject." 2022 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, 2022. Vats, Anuja, et al. "This changes to that: Combining causal and non-causal explanations to generate disease progression in capsule endoscopy." ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023. Zhao, Ziwei, et al. "Generating Counterfactual Images: Towards a C2C-VAE Approach." (2022). ## Not directly related to XAI Lu, Jinghui, et al. "A rationale-centric framework for human-in-the-loop machine learning." arXiv preprint arXiv:2203.12918 (2022). Hagos, Misgina Tsighe, Kathleen M. Curran, and Brian Mac Namee. "Identifying Spurious Correlations and Correcting them with an Explanation-based Learning." arXiv preprint arXiv:2211.08285 (2022). Ye, Xiaomeng, et al. "Applying the case difference heuristic to learn adaptations from deep network features." arXiv preprint arXiv:2107.07095 (2021). Leofante, Francesco, and Alessio Lomuscio. "Towards Robust Contrastive Explanations for Human-Neural Multi-agent Systems." Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems. 2023. ## Using for Evaluation Wilkerson, Zachary, David Leake, and David Crandall. "Leveraging SHAP and CBR for Dimensionality Reduction on the Psychology Prediction Dataset." (2022). ## Survey Papers Bodria, Francesco, et al. "Benchmarking and survey of explanation methods for black box models." Data Mining and Knowledge Discovery (2023): 1-60. Nimmy, Sonia Farhana, et al. "Explainability in supply chain operational risk management: A systematic literature review." Knowledge-Based Systems 235 (2022): 107587. Keane, Mark T., et al. "Twin systems for deepcbr: A menagerie of deep learning and case-based reasoning pairings for explanation and data augmentation." arXiv preprint arXiv:2104.14461 (2021). Kenny, Eoin M., et al. "Post-hoc explanation options for XAI in deep learning: The Insight centre for data analytics perspective." Pattern Recognition. ICPR International Workshops and Challenges: Virtual Event, January 10–15, 2021, Proceedings, Part III. Springer International Publishing, 2021. Poché, Antonin, Lucas Hervier, and Mohamed-Chafik Bakkay. "Natural Example-Based Explainability: a Survey." (2023). Holmberg, Lars, Paul Davidsson, and Per Linde. "Mapping Knowledge Representations to Concepts: A Review and New Perspectives." arXiv preprint arXiv:2301.00189 (2022). Nazir, Sajid, Diane M. Dickson, and Muhammad Usman Akram. "Survey of explainable artificial intelligence techniques for biomedical imaging with deep neural networks." Computers in Biology and Medicine (2023): 106668. Keane, Mark T., et al. "If only we had better counterfactual explanations: Five key deficits to rectify in the evaluation of counterfactual xai techniques." arXiv preprint arXiv:2103.01035 (2021). Verma, Sahil, et al. "Counterfactual explanations and algorithmic recourses for machine learning: A review." arXiv preprint arXiv:2010.10596 (2020). – This is a great example of how tablar data is a REALLY important distinction ## Citing in Context of Counterfactuals Wang, Zijie J., et al. "GAM Coach: Towards Interactive and User-centered Algorithmic Recourse." Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 2023. Aguilera-Ventura, Carlos, et al. "Counterfactual Reasoning via Grounded Distance." Proceedings of the International Conference on Principles of Knowledge Representation and Reasoning. Vol. 19. No. 1. 2023. Wang, Chong, et al. "Learning Support and Trivial Prototypes for Interpretable Image Classification." arXiv preprint arXiv:2301.04011 (2023). Booth, Serena, et al. "Bayes-trex: a bayesian sampling approach to model transparency by example." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 35. No. 13. 2021. Höltgen, Benedikt, et al. "Deduce: generating counterfactual explanations at scale." eXplainable AI approaches for debugging and diagnosis.. 2021. Yang, Linyi, et al. "Exploring the efficacy of automatically generated counterfactuals for sentiment analysis." arXiv preprint arXiv:2106.15231 (2021).
Our Categorisations of Kenny & Keane's Citations (part 2)
... Höltgen, Benedikt, et al. "Deduce: Generating counterfactual explanations efficiently." arXiv preprint arXiv:2111.15639 (2021). Schemmer, Max, et al. "Towards Meaningful Anomaly Detection: The Effect of Counterfactual Explanations on the Investigation of Anomalies in Multivariate Time Series." arXiv preprint arXiv:2302.03302 (2023). Guidotti, Riccardo. "Counterfactual explanations and how to find them: literature review and benchmarking." Data Mining and Knowledge Discovery (2022): 1-55. Yang, Linyi, et al. "Generating plausible counterfactual explanations for deep transformers in financial text classification." arXiv preprint arXiv:2010.12512 (2020). Delaney, Eoin, Derek Greene, and Mark T. Keane. "Uncertainty estimation and out-of-distribution detection for counterfactual explanations: Pitfalls and solutions." arXiv preprint arXiv:2107.09734 (2021). Cho, Soo Hyun, and Kyung-shik Shin. "Feature-Weighted Counterfactual-Based Explanation for Bankruptcy Prediction." Expert Systems with Applications 216 (2023): 119390. Huang, Qinhua, and Weimin Ouyang. "Improving Causality Explanation of Judge-View Generation Based on Counterfactual." International Conference on Intelligent Computing. Singapore: Springer Nature Singapore, 2023. Warren, Greta, Barry Smyth, and Mark T. Keane. "“Better” Counterfactuals, Ones People Can Understand: Psychologically-Plausible Case-Based Counterfactuals Using Categorical Features for Explainable AI (XAI)." International conference on case-based reasoning. Cham: Springer International Publishing, 2022. Albini, Emanuele, et al. "On the Connection between Game-Theoretic Feature Attributions and Counterfactual Explanations." arXiv preprint arXiv:2307.06941 (2023). Jeanneret, Guillaume, Loïc Simon, and Frédéric Jurie. "Diffusion models for counterfactual explanations." Proceedings of the Asian Conference on Computer Vision. 2022. Delaney, Eoin, et al. "Counterfactual explanations for misclassified images: How human and machine explanations differ." arXiv preprint arXiv:2212.08733 (2022). Warren, Greta, et al. "Explaining Groups of Instances Counterfactually for XAI: A Use Case, Algorithm and User Study for Group-Counterfactuals." arXiv preprint arXiv:2303.09297 (2023). Pawelczyk, Martin, Lea Tiyavorabun, and Gjergji Kasneci. "Decomposing Counterfactual Explanations for Consequential Decision xMaking." arXiv preprint arXiv:2211.02151 (2022). Jiang, Junqi, et al. "Formalising the robustness of counterfactual explanations for neural networks." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 37. No. 12. 2023. Warren, Greta, Ruth MJ Byrne, and Mark T. Keane. "Categorical and continuous features in counterfactual explanations of AI systems." Proceedings of the 28th International Conference on Intelligent User Interfaces. 2023. Zhou, Yilun. "Iterative Partial Fulfillment of Counterfactual Explanations: Benefits and Risks." arXiv preprint arXiv:2303.11111 (2023). Fahse, Tobias Benjamin, Ivo Blohm, and Benjamin van Giffen. "Effectiveness of Example-Based Explanations to Improve Human Decision Quality in Machine Learning Forecasting Systems." (2022). Labaien Soto, Jokin, Ekhi Zugasti Uriguen, and Xabier De Carlos Garcia. "Real-Time, Model-Agnostic and User-Driven Counterfactual Explanations Using Autoencoders." Applied Sciences 13.5 (2023): 2912. Delaney, Eoin, Derek Greene, and Mark T. Keane. "Instance-based counterfactual explanations for time series classification." International Conference on Case-Based Reasoning. Cham: Springer International Publishing, 2021. Na, Seung-Hyup, Woo-Jeoung Nam, and Seong-Whan Lee. "Toward practical and plausible counterfactual explanation through latent adjustment in disentangled space." Expert Systems with Applications 233 (2023): 120982. Patil, Anish, et al. "Utilization of GAN for Automatic Evaluation of Counterfactuals: Challenges and Opportunities." Pawelczyk, Martin, et al. "Carla: a python library to benchmark algorithmic recourse and counterfactual explanation algorithms." arXiv preprint arXiv:2108.00783 (2021). Salazar, Sebastian, Samuel Denton, and Ansaf Salleb-Aouissi. "Counterfactual Explanations for Support Vector Machine Models." arXiv preprint arXiv:2212.07432 (2022). ## Cited in Context of CBR Horta, Vitor AC, and Alessandra Mileo. "Generating Local Textual Explanations for CNNs: A Semantic Approach Based on Knowledge Graphs." International Conference of the Italian Association for Artificial Intelligence. Cham: Springer International Publishing, 2021. Ye, Xiaomeng, et al. "Learning adaptations for case-based classification: A neural network approach." Case-Based Reasoning Research and Development: 29th International Conference, ICCBR 2021, Salamanca, Spain, September 13–16, 2021, Proceedings 29. Springer International Publishing, 2021.
Our Categorisations of Kenny & Keane's Citations (part 3)
... Du, Haiwen, et al. "Can We Transfer Noise Patterns? An Multi-environment Spectrum Analysis Model Using Generated Cases." arXiv preprint arXiv:2308.01138 (2023). Leake, David, et al. "Enhancing Case-Based Reasoning with Neural Networks." Compendium of Neurosymbolic Artificial Intelligence. IOS Press, 2023. 387-409. ## Positing Use for Semi-factuals in User Testing Salimi, Pedram. "Addressing Trust and Mutability Issues in XAI utilising Case Based Reasoning." ICCBR Doctoral Consortium 2022 1613 (2022): 0073. Mueller, Shane, et al. "Authoring Guide for Cognitive Tutorials for Artificial Intelligence: Purposes and Methods." (2021). Hoffman, Robert R., et al. "Evaluating machine-generated explanations: a “Scorecard” method for XAI measurement science." Frontiers in Computer Science 5 (2023): 1114806. Mueller, Shane T., et al. "A Computational Cognitive Model of Informative and Persuasive Explanations of Artificial Intelligence Systems." ## Discuss Semi-Factuals Montenegro, Helena, et al. "Privacy-preserving case-based explanations: enabling visual interpretability by protecting privacy." IEEE Access 10 (2022): 28333-28347. Kenny, Eoin M., and Mark T. Keane. "Explaining Deep Learning using examples: Optimal feature weighting methods for twin systems using post-hoc, explanation-by-example in XAI." Knowledge-Based Systems 233 (2021): 107530. Montenegro, Helena, Wilson Silva, and Jaime S. Cardoso. "Privacy-preserving generative adversarial network for case-based explainability in medical image analysis." IEEE Access 9 (2021): 148037-148047. Kenny, Eoin M., et al. "Explaining black-box classifiers using post-hoc explanations-by-example: The effect of explanations and error-rates in XAI user studies." Artificial Intelligence 294 (2021): 103459. Warren, Greta, Mark T. Keane, and Ruth MJ Byrne. "Features of Explainability: How users understand counterfactual and causal explanations for categorical and continuous features in XAI." arXiv preprint arXiv:2204.10152 (2022). Carraro, Diego, and Kenneth N. Brown. "CouRGe: Counterfactual Reviews Generator for Sentiment Analysis." Irish Conference on Artificial Intelligence and Cognitive Science. Cham: Springer Nature Switzerland, 2022. Leake, David, Zachary Wilkerson, and David Crandall. "Extracting case indices from convolutional neural networks: A comparative study." International Conference on Case-Based Reasoning. Cham: Springer International Publishing, 2022. Mertes, Silvan, et al. "Alterfactual Explanations--The Relevance of Irrelevance for Explaining AI Systems." arXiv preprint arXiv:2207.09374 (2022). ## Other Situ, Xuelin. Explaining the Outcomes of Deep Learning Models. Diss. Monash University, 2022. Brown, Katherine Elizabeth. Evaluating, Explaining, and Utilizing Model Uncertainty in High-Performing, Opaque Machine Learning Models. Diss. Tennessee Technological University, 2023. Jouis, Gaëlle. Explicabilité des modèles profonds et méthodologie pour son évaluation: application aux données textuelles de Pôle emploi. Diss. Nantes Université, 2023. Yang, Linyi. Deep Neural Approach for Financial Analysis. Diss. University College Dublin, 2021. Owens, Jelani. CLCS: A Convolutional Learning Classifier System for the Surrogate Model in Higher-Dimensional Space. Diss. North Carolina Agricultural and Technical State University, 2022. Hong, Xianbin, et al. "Dual-Track Lifelong Machine Learning-Based Fine-Grained Product Quality Analysis." Applied Sciences 13.3 (2023): 1241. Buijsman, Stefan. "Defining explanation and explanatory depth in XAI." Minds and Machines 32.3 (2022): 563-584. Nimmy, Sonia Farhana, et al. "Interpreting the antecedents of a predicted output by capturing the interdependencies among the system features and their evolution over time." Engineering Applications of Artificial Intelligence 117 (2023): 105596. Wen, Bingyang, et al. "Causal-TGAN: Causally-Aware Synthetic Tabular Data Generative Adversarial Network." (2021). ## Psychology Literature Sarasvathy, Saras D. "Even-if: Sufficient, yet unnecessary conditions for worldmaking." Organization Theory 2.2 (2021): 26317877211005785.
Decision
Accept (poster)