Counterfactually Comparing Abstaining Classifiers

Abstaining classifiers have the option to abstain from making predictions on inputs that they are unsure about. These classifiers are becoming increasingly popular in high-stakes decision-making problems, as they can withhold uncertain predictions to improve their reliability and safety. When evaluating black-box abstaining classifier(s), however, we lack a principled approach that accounts for what the classifier would have predicted on its abstentions. These missing predictions matter when they can eventually be utilized, either directly or as a backup option in a failure mode. In this paper, we introduce a novel approach and perspective to the problem of evaluating and comparing abstaining classifiers by treating abstentions as missing data. Our evaluation approach is centered around defining the counterfactual score of an abstaining classifier, defined as the expected performance of the classifier had it not been allowed to abstain. We specify the conditions under which the counterfactual score is identifiable: if the abstentions are stochastic, and if the evaluation data is independent of the training data (ensuring that the predictions are missing at random), then the score is identifiable. Note that, if abstentions are deterministic, then the score is unidentifiable because the classifier can perform arbitrarily poorly on its abstentions. Leveraging tools from observational causal inference, we then develop nonparametric and doubly robust methods to efficiently estimate this quantity under identification. Our approach is examined in both simulated and real data experiments.

Paper

References (73)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer 9d5v4/10 · confidence 3/52023-06-15

Summary

This paper outlines a method for estimating the counterfactual performance of an abstain classifier on cases where it abstained (what would have happened if it had not abstained?). Using tools from causal inference and the potential outcomes framework in particular, they cast this is a missing data problem, and provide estimators for the relevant quantity, proving identifiability under standard conditions and showing convergence. Experimentally, they use semi-synthetic setups to show that this estimator produces good confidence intervals.

Strengths

- This paper is clearly presented and technically sound - the notion of connecting abstaining classifiers to missing data problems is interesting and I believe novel - the demonstration of the applicability of the missing at random (conditional on X) assumption to this problem is insightful and helpful - experiments are clear and I believe the practical effectiveness of this method

Weaknesses

- Motivation: I'm not convinced that the problem this paper solves is a realistic one - the examples seem fairly contrived to me. I'm not sure why any API would be provided in the form described in 1.1 (with a free abstaining tier), and examples in the appendix all assume an importance of the "hidden" predictions which doesn't seem super realistic. For instance, in A.3 it doesn't seem well motivated to want to evaluate a classifier's fairness based on its hidden predictions, as those do not actually result in impacts or harms on downstream populations. - Novelty: I think that the connection of this problem to causality + missing data is novel, but I'm not sure there's much novel insight here. I do think the argument around the missing at random assumption is good (Assumption 2.1) as stated previously, but there isn't much else in this paper which is specific to the abstaining classifier problem. Smaller notes: line 4: "stake" -> stakes line 50: unclear what "provably unavoidable" means line 213: is \mu_0 defined anywhere? line 225: the word "either" here is confusing - probably want to use "each" instead line 324: the word "nuisance" is used many times in this section - is what it refers to defined? line 397: might be worth defining the Gini impurity more clearly and explaining why this is a good metric

Questions

- what are some other ways that missing data/causality tools can be adapted to this problem specifically? for instance, are there implications around confidence-based rejection that can be leveraged for estimation methods? - in some cases, the underlying prediction may not even be well defined when R = 1 - for instance in cascading models where computation is only proceeded with after the rejection decision is made (if necessary). how do these models fit into the conceptualization of this paper?

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

3 good

Contribution

2 fair

Limitations

n/a

Reviewer 9p416/10 · confidence 3/52023-07-05

Summary

This paper proposes to compare black-box abstaining classifiers (neither the base classifier nor the abstention mechanism is known to the evaluator) by the counterfactual score, that is the expected score of the abstaining classifier had it not been given the option to abstain. They prove that this quantity is identifiable under two standard conditions, that are missing at random and positivity, and then develop nonparametric and doubly robust methods to estimate it. Experiments on simulated data and CIFAR-100 are conducted.

Strengths

This paper is well-organized and clearly written. Comparing abstaining classifiers is an important problem in machine learning and the counterfactual score proposed in this paper is a new evaluation metric that is relevant in practice.

Weaknesses

1. As stated in the paper, the counterfactual score is unidentifiable if abstentions are deterministic which includes a fair amount of abstention methods in the literature. 2. It would be better to briefly discuss some related works on abstention instead of just referring to surveys.

Questions

1. In the learning-to-defer setting, where instead of rejecting an example, the example can be deferred to an expert and let that expert make the prediction, does the counterfactual score still make sense? minor: 1. There is a redundant 'is' at the end of line 420.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

Yes.

Reviewer fENC7/10 · confidence 3/52023-07-05

Summary

The authors focus on the problem of evaluating abstaining classifiers while also taking account their counterfactual predictions had they not abstained on certain data points. As an evaluation metric they consider the expected accuracy of the classifiers over both their actual and counterfactual predictions, which they define as the counterfactual score. They provide the conditions under which the counterfactual score is identifiable and also propose a doubly robust estimator to compute the counterfactual score of an abstaining classifier. Finally, they evaluate their estimator with simulations and experiments on real data.

Strengths

The idea of considering the counterfactual predictions of the classifier had it not abstained for evaluation is conceptually quite interesting and also could have a significant impact in scenarios when the expert, for example, who predicts when the classifier abstains is also uncertain and would still choose the classifier’s prediction. The paper is very nicely written, well structured and organised, well motivated and convincing. The experimental evaluation includes both simulations and real data experiments, the setup and the results are clearly explained and also the code is provided for reproducibility.

Weaknesses

There see to be in general no major weaknesses. It is a bit confusing the pointer to table 1 in line 86, where the notation has not yet been introduced. It would have been nicer perhaps not to include notation at that point or explain the notation in the caption. Moreover, the captions of the tables should have been above the tables according to the author instructions. Typos: -Line 87: ‘of significant interest’ -Line 367: there seem to be a missing verb -Line 420: the sentence does not make sense, it seems that some part is missing.

Questions

N/A.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

The authors adequately address the limitations of their work.

Reviewer paQd5/10 · confidence 4/52023-07-06

Summary

The paper proposes a new way to evaluate counterfactually, abstain classifiers. By considering the task of selective classification, the authors reformulate the problem into a Missing data problem which can thus be seen as a causal inference (Counterfactual) problem. The authors also state the corresponding assumptions needed for the above to be identifiable and propose a doubly robust estimator for their proposed metric.

Strengths

- The paper proposes a new view on evaluation of abstain classifiers through the lens of Rubins Counterfactuals - The paper clearly states the assumptions for the problem to be identifiable - The paper clearly formulates the approach of doubly robust estimations and how it can be used in their setting for abstain classifier comparisons.

Weaknesses

- My biggest concern of the paper is the motivation. As stated in their example 1.1 the example they have given is quite contrived and not really applicable in most cases. I believe much stronger motivating examples would be useful. If the authors would clearly give me better examples (in addition to the appendix ones) that would greatly help me position this paper. - Secondly, I believe the experiments are not clearly written out to me. Could the authors tell me how exactly this metric is preferred over simply looking at the accuracy of the selected labels and the inaccuracy of the non-selected labels? I might have misunderstood this part, so if the authors could help me clarify this part it would be very helpful. The authors mention that this is an "inverse" problem, but not sure if I understood this part. - Lastly, I wonder if the authors have thought of implementing the second point mentioned in this review. I am confused about why there are no other baselines. I would have thought that the second point would constitute a simple baseline.

Questions

see in weakness I am more than happy to raise my score if the above are clarified.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

Yes the authors have addressed limitations of their work

Reviewer 9d5v2023-08-11

Response

Thanks for the rebuttal. - I appreciate the points in the main response re: motivation and I think that the more fully fleshed out argument is more compelling than what's in the paper currently. I think it's probably important to include a more in-depth motivation section in a paper like this. - I agree that there is novelty in the abstaining classifier-missing data connection and it's possible I underrated it's value in my original review. - I agree that the counterfactual score is defined in the cascading case - my fault for stating it poorly in the original review. In general, on re-reading my review and scanning the paper again, I'm inclined to re-consider my score, and will do so as I discuss with the other reviewers.

Authorsrebuttal2023-08-11

Author Response

Thank you for your encouraging response. We will further flesh out the motivation section of the paper by incorporating the additional points from our rebuttal. We also appreciate you for bringing up the cascading model example, and we will incorporate our response in the paper. We hope that these responses help you reconsider your assessment of our work.

Reviewer 9p412023-08-17

Thank you for your response. It addressed some of my concerns. I have also read the other reviews and rebuttals and would like to keep my initial rating.

Reviewer paQd2023-08-18

Thanks for the reply

First of all i would like to thank the authors for the reply. The motivation part has been partially justified for me, even though i remain skeptical, I believe there is a chance this might become useful in the future. In terms of the metric, I do not understand exactly why the metric by Condessa et al is not applicable in "evaluating" abstain classifier? To me it seems like the most natural and basic way to evaluate a selective classifier. Could the authors please let me know what the proposed score does, that current methods cant exactly. I would like to hear an exact and precise example please. Also I dont understand this sentence. So what is the point then? I am really confused now, could you please clarify this part. "These experiments are NOT meant to provide further justifications on why the counterfactual score is a viable alternative as a metric to existing ones; they are simply meant to show that our method for estimating the counterfactual score is sensible and works well in practice". As it stands, i cannot raise my score and hope that the authors can convince me otherwise.

Authorsrebuttal2023-08-19

Further clarification

> In terms of the metric, I do not understand exactly why the metric by Condessa et al is not applicable in "evaluating" abstain classifier? To me it seems like the most natural and basic way to evaluate a selective classifier. Could the authors please let me know what the proposed score does, that current methods cant exactly. We did not say anywhere that Condessa et al.’s metric is not applicable in evaluating an abstaining classifier. We elaborate on when the counterfactual score is preferred over Condessa et al.’s score in lines 89–103 in our paper, including the following line: _While [Condessa et al.’s] view is relevant to the training of abstention rules, it is at odds with black-box settings where the underlying predictions may still be executed even when the method abstains, motivating the counterfactual score._ The questions of (1) whether a score is natural and (2) what “current methods can’t do” are separate. 1. Our whole intro is devoted to justifying that the counterfactual score, as we defined it in our paper, can be useful (rather than Condessa et al.’s). Both metrics can be viable metrics for evaluating abstaining classifiers, but they simply evaluate the classifiers in a different manner. The two scores can be useful in different scenarios, and we focused on scenarios in which our proposed metric can be useful. 2. Condessa et al. (or any other work, to our knowledge) do not discuss the black-box setting that requires a _statistical estimation_ approach, as they already assume full knowledge of whether the classifier would have classified each input correctly or not. But this is not known in any of the black-box settings we describe. Both Condessa et al.’s score and the counterfactual score require how accurate the classifier _would have been_ on its abstentions, and we do not know this in the black-box scenario like Example 1.1 (we have no clue about whether an API would get a 100% or 0% accuracy on inputs that it chose to abstain from making predictions). This unknown quantity is really the “counterfactual,” and thus we devote Sections 3 and 4 of our paper to discuss how we can _estimate_ the counterfactual score in the black-box setting. No other "current method" can estimate the counterfactual score (because it was never defined formally), and we further mention in Appendix A.3 that our method can further estimate Condessa et al.’s score in the black-box setting. > I would like to hear an exact and precise example please. To give a simplified example, suppose that we compare two abstaining classifiers on 100 data points, whose inputs are sampled uniformly on $[-1, 1] \times [-1, 1]$. Suppose that classifier A achieves a 1.0 accuracy on the left half of the input space ($x_1 < 0$) but a 0.8 accuracy on the right half ($x_1 \geq 0$). It abstains at an 80% rate on the right half, while it does not abstain at all on the left half. On the left half, the classifier makes 50/50 correct predictions on the left half; on the right half, it makes 8/10 correct predictions and 40 abstentions (for which it would have been correct 80% of the time). Recall the definitions from Appendix A.3. The counterfactual score of classifier A is 58/60 * 0.6 + 32/40 * 0.4 = 0.9, whereas Condessa et al.’s score for this classifier is 58/60 * 0.6 + (1 - 32/40) * 0.4 = 0.66. Note that the classifier’s selective score is 58/60=0.97 while its coverage is 60/100=0.6. Next, suppose that classifier B is the same as classifier A, except that it achieves a 0.6 accuracy on the right half. Then, classifier B’s counterfactual score would be 0.8, lower than classifier A, whereas Condessa et al.’s score for B would be 0.82. If the evaluator needs to access the classifier’s hidden predictions on the right half, they would prefer classifier A, which has a higher accuracy had no abstentions were made. In Example 1.1, the right half would correspond to the inputs that the free-trial API chose to abstain on. > Also I dont understand this sentence. So what is the point then? I am really confused now, could you please clarify this part. "These experiments are NOT meant to provide further justifications on why the counterfactual score is a viable alternative as a metric to existing ones; they are simply meant to show that our method for estimating the counterfactual score is sensible and works well in practice". Because we developed an estimation approach, we need experiments to validate whether the estimator actually estimates the target quantity well on simulated (and real) data. We note that it is _not_ trivial whether this score can be estimated well in practice, given that it is an unknown counterfactual quantity. The real data experiments illustrate different comparison scenarios and what the estimated counterfactual score would be. We hope that these responses clarify your concerns.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC