Improving Decision Sparsity

Summary

The paper extends the notion of decision sparsity called the sparse explanation value (SEV). Cluster-based and tree-based SEV are introduced, as well as some algorithms to optimise the decision sparsity are considered. The core of the paper -- SEV -- is defined as the number of factors that need to be changed to a reference values in order to change the decision.

Strengths

The problem considered is important for real (e.g., medical, criminology applications). The example (Table 2) is very helpful to understand the problem. The approach is mathematically sound and in general well-described.

Weaknesses

The method is promising but its current version is hardly scalable.

Questions

Line 85: "humans have no intuition for why a point belongs to one class or the other". I cannot completely agree with the statement, and it would be helpful to provide some examples. For me, on the contrary, medical doctors have often an intuition how to classify a patient, however, they do not always know how to explain their intuition. How the sparse explanation for the sample x (mentioned on line 171) is formally defined? Is the number of features of x (original) and x (sparse) different?

Rating

7

Confidence

4

Soundness

4

Presentation

3

Contribution

3

Limitations

The method is not scalable.

Summary

The authors build on top of the Sparse Explanation Value approach by Sun et al and provide improvements in terms of closeness and credibility.

Strengths

Sensible problem, well presented solution.

Weaknesses

The main limitation I can see in the work is its very incremental nature with respect to the approach by Syn et al.: going from one negative reference point to a set by clustering negatives (cluster-based SEV) is a trivial extension, while tree-based SEV only works if the underlying model is (or can be approximated as) a decision tree, substantially restricting the applicability of the approach. Additionally, the superiority of sparsity with respect to distance in terms of acceptability for humans is intuitive but not always guaranteed. When dealing with recourse, for instance, which is the setting used in the experiments, the main problem is the cost of the change, and slightly modifying two features could be less expensive than modifying a single one by a larger value. This should be better discussed in the paper.

Questions

How can you turn a DT leaf into a reference point when dealing with continuous features? what is the actual value of the continuous variables in the leaf?

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

The approach is very incremental, and the main extension substantially restricts the applicability of the method. The rebuttal of the authors did shade additional light on the novelty of the contribution.

Summary

This paper proposes several ways to create closer, sparser, and more credible explanations for the SEV, along with two optimizing models. The results of the experiments on various datasets support the paper's claims.

Strengths

1. Before reading this paper, I was unfamiliar with the sparse decision field. However, this paper is well-written and enjoyable to read. 2. I think this paper is quite creative by simultaneously considering closeness, sparsity, and credibility. 3. The comprehensive experiments address most of the claims they proposed in the introduction section.

Weaknesses

It could be better to provide the complexity analysis and the analysis of time expenditure for different variants of SEV. For example, the computational benefits of using tree-based SEV.

Questions

1. In Table 1, what do the gray numbers represent? 2. In line 199, there are duplicate "and"s.

Rating

8

Confidence

2

Soundness

4

Presentation

4

Contribution

4

Limitations

I'm concerned about the depth of the tree if the model applies to large-scale data. In section A, the number of observations is up to 100K. If we encounter a much larger data set, the complexity of the model would exponentially increase with the depth of the tree. That's why I'm interested in time analysis.

Discussion

Hi all, the authors have provided detailed responses to the reviews. Please read the responses, and take advantage of the reviewer-author discussion period to ask for additional clarifications if needed. In particular, reviewer bvrF has given the lowest score due to a number of issues she/he identified with the submission. The authors addressed these issues in their response. Does the reviewer agree with the response?

Thanks for your answer

Thanks for your detailed feedback, I appreciate the clarifications and your arguments in favour of the novelty of the contribution, I encourage you to better clarify these aspects in the manuscript. This said, I am happy to raise my score to a borderline accept.