Reject option models comprising out-of-distribution detection

Summary

The authors consider 3 rejection models in the OOD detection setting and establish theorems about the optimal rules for these models. Due to the commonly used metrics that may partially only focus on either OOD detection or misclassification, they proposed a double-score method to consider both aspects.

Strengths

* The authors extend the rejection in the non-OOD setting to the OOD detection setting. * The 3 different rejection models share similar optimal strategies, and those strategies outperform other baselines. * Since previous works only focus on either OOD detection or misclassification, the authors proposed a novel double-score method to consider both aspects.

Weaknesses

* Now that we know all the distributions, including OOD and ID, why put effort into eq 3? Can’t we just treat augmented data of OOD + ID as the new collection with $K + 1$ "inlier" classes, then follow the regular rejection analysis under the closed-world assumption? * The optimal rules heavily rely on the clear information (the distribution in the theorem or estimated from the sample) on OOD data. For example, the prior $\pi$ in the bounded precision-recall rejection model, the conditional probability (of OOD) in the bounded TPR-FPR rejection model, or even both in the cost-based model. However, it is challenging to access OOD data and estimate these probabilities in real-world applications within an open-world setting. This raises concerns about the practical guidance provided by these optimal rules. * I have a reservation about the constraints in equation 8 or the setup of problem 1. In practice, it is more likely the threshold or the selective function is determined by only one constraint (even in your proof of theorem 2, there is nowhere about the constraint $\rho(c)$). For example, once the $\lambda$ is determined by one constraint, it does not necessarily satisfy another one, which is similar to the trade-off between the type I and type II errors or Neyman-Pearson classification. A similar argument applies to Problem 2. * Optimal rules: the authors claim the optimal rules related to the Bayes classifier (lines 126, 170), but from theorems, it holds under the condition that the given $(h, c)$ is optimal. If we cannot find the optimal $h$, can we still say the Bayes classifier is optimal?

Questions

* Line 137: typos in "Let $g(x)$ …. samples" $\rightarrow$ $p_O(x)/p_I(x)$... ratio of OOD and ID samples. * Theorem 3: Do you mean any $h$ is optimal even if it is not the Bayes classifier? The optimality looks like is conditional on the given $h$, how could you guarantee the picked $h$ associated with the corresponding $c$ simultaneously achieve to the minimizer? * Line 157: What does this notation mean: (TPR, $\phi$)? * Eq 12: Since $h^*=h_B$, do you mean $r_B(x)$ instead of $r(x)$? * Line 301 (FPR@TPRx, TNR@TPRx, CCR@FPRx): These metrics also control one metric @ percentage x and then compared to another one, depending on their definitions of positive/negative for the ID and OOD, which is similar to the idea of your proposed problem 1/2, like controlling two metrics (actually can only achieve one constraint in general) and compare the selective risk. * Line 441: Could you explain why $\min_{q\in \mathcal{Y}}R_x(q)$ is in this form (are you saying the smallest risk is always achieved by making predictions even for OOD data?), while later you use $r(x)$ instead of $r_B(x)$ in $\min_{q\in \mathcal{Y}}R_x(q)$ (1st equality below line 443). * Line 449: You miss the subscript $I$ under $p$.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

See my concerns and questions above.

Summary

This paper provides a unified viewpoint of reject model evaluation metrics under OOD-ness. Despite different performance metrics being used in practice, they can all be factorized into both the misclassification prediction on ID datapoints as well as the discrimination performance between ID and OOD. The experimental section provides some evidence that this proposed two score decomposition can provide competitive OOD detection and input rejection performance.

Strengths

- OOD detection and classifying with a reject option are both timely and important issues in trustworthy machine learning. ML algorithms need to be aware of their limitations and signal if they are deployed on previously unseen data. - The unified framework / viewpoint makes sense and yields an interesting decomposition result showing that the optimal uncertainty score is a function of both misclassification prediction on ID datapoints as well as distinction performance between ID and OOD. Although this appears like an intuitive desiderata for a prediction uncertainty score, explicitly tying it back to the different rejection models seems new.

Weaknesses

- The paper assumes that the OOD distribution is known. This is a strong assumption and severely limits the applicability of the approach in practical settings where knowing or explicitly estimating this distribution is often prohibitive. - Even though the paper does provide a nice unification of three performance measures for reject option models in an OOD setup (cost-based rejection, TPR-FPR rejection, precision-recall rejection), these metrics are not new and routinely used in existing applications. To me it seems like this kind of unified viewpoint would be better suited for a survey-style paper. - Section 3 could have been motivated better. To me, it appears a bit sudden without much justification or transition from the previous section. - Section 3.1 could have significantly profited from a figure showcasing the 1D Gaussian example for added intuition. - While the optimal uncertainty score is unified across error models, the experimental section showing the efficacy of the double score method is very lackluster. Details about models, training methods, and hyper-parameters area all missing (and also not documented in the appendix). Results are also based on a single run which makes it impossible to judge the statistical significance of the presented results. Moreover, the considered datasets do not all share the same sample shapes (e.g. MNIST vs CIFAR-10) which makes me wonder how OOD scores were obtained here in the first place. Real life shifts, like from the WILDS dataset collection would have been a better place to assess OOD-ness of samples. I am not convinced by this evaluation. ### Post-rebuttal I have read the author's rebuttal and found that although many of my concerns were adequately addressed, the experimental documentation issue remained unaddressed. I increased my score only slightly as a result.

Questions

- Can you add additional motivation for Section 3? - Can you provide more experimental details?

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

Weaknesses of the approach are not adequately discussed in the paper. See weaknesses and questions above.

Summary

This paper proposes three reject option models and introduces double-score OOD methods that consistently outperform state-of-the-art methods. The authors also propose novel evaluation metrics for comprehensive and reliable assessment of OOD methods. The proposed metrics simultaneously evaluate the classification performance on the accepted ID samples and guarantee the performance of the OOD/ID discriminator, either via constraints in TPR-FPR or Precision-Recall pair. The authors argue that setting these extra parameters is better than using the existing metrics that provide incomplete if used separately, or inconsistent, if used in combination, view of the evaluated methods. Overall, the paper's contributions provide a significant improvement in OOD detection and evaluation.

Strengths

The paper proposes three reject option models for OOD setups, which extend the standard reject option models. These models define the notion of an optimal OOD selective classifier and establish that all the proposed models, despite their different formulations, share a common class of optimal strategies. This is an original and creative approach to the problem of OOD detection, and the proposed models are well-motivated and clearly explained. The paper introduces double-score OOD methods that leverage uncertainty scores from two chosen OOD detectors: one focused on OOD/ID discrimination and the other on misclassification detection. The paper proposes novel evaluation metrics derived from the definition of the optimal strategy under the proposed OOD rejection models. These metrics provide a comprehensive and reliable assessment of OOD methods. Overall, the paper is well-written and easy to follow, with clear explanations of the proposed models, methods, and evaluation metrics.

Weaknesses

One weakness of the paper in the experimental results is that the dataset used in the experiments is relatively small, and the proposed methods do not show significant advantages over the baseline. This raises questions about the generalizability of the proposed methods to larger and more diverse datasets. Additionally, the paper could benefit from a more detailed analysis of the limitations of the proposed methods and how they can be improved in future work.

Questions

1. The dataset used in the experiments of the paper is relatively small, and the proposed methods do not show significant advantages over the baseline in terms of experimental results. It would be helpful for the quality of the paper if the authors could provide more compelling experimental results or demonstrate the effectiveness of the proposed methods on larger datasets.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors did not explicitly discuss the limitation of the proposed method.

Summary

This paper addresses out-of-distribution detection (ID/OOD discrimination) and misclassification detection (selection classification). The authors introduce double-score OOD methods that leverage uncertainty scores from OOD detector and misclassification detector. For evaluation metric, this paper proposes to use PR curve.

Strengths

(1) Existing works mainly focus on OOD detection, which this paper simultaneously considers OOD and selection classification. This is more practical and useful for high risk applications. (2) Theoretical analysis of OOD detection and selective classification is valuable.

Weaknesses

(1) The analysis of Bayes-optimal OOD selective classifier as well as the Bayes-optimal misclassification selective classifier has also been investigated in [1]. The authors are suggested to demonstrate the difference. (2) The experiments are insufficient, only evaluating on cifar-10 and mnist. Results on CIFAR-100 is valuable. (3) What is the advantage of PR curve over AURC (risk-coverage) curve [2] for evaluating rejection model? [1] Narasimhan, H., Menon, A. K., Jitkrittum, W., & Kumar, S. (2023). Learning to reject meets OOD detection: Are all abstentions created equal?. arXiv preprint arXiv:2301.12386. [2] Geifman, Y., & El-Yaniv, R. (2017). Selective classification for deep neural networks. Advances in neural information processing systems, 30.

Questions

A recent work [3] also addresses the selective classification and OOD detection at the same time. it is worth mentioning it. [3] Zhu, F., Cheng, Z., Zhang, X. Y., & Liu, C. L. (2023). OpenMix: Exploring Outlier Samples for Misclassification Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 12074-12083).

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

n/a

Summary

The paper presents a formal analysis of three distinct models for classifiers with a reject option in the presence of out-of-distribution inputs at test time. All three models, viz. 1) cost-based rejection, 2) bounded TPR-FPR, and 3) bounded precision-recall, share the same form of optimal selective classifier (see Section 2.4). This selective classifier consists of the Bayes in-distribution classifier $h_B(x)$ (which is optimal for a given in-distribution), and a selection function score that is a linear combination of the conditional risk $r(x)$ and the likelihood-ratio of OOD to ID $g(x)$. Based on this analysis, the authors point out the limitations of current OOD detection methods which only have a single score, and instead propose double-score OOD detection methods which can focus on both mis-classification detection and ID/OOD separation. Using a concrete synthetic example, they discuss the limitations of existing metrics such as AUROC and AUPR in evaluating selective classifiers in the OOD setting. They propose a novel metric (one each for the bounded TPR-FPR and bounded precision-recall models) which calculates the selective risk, subject to a given minimum TPR and maximum FPR (or a given minimum recall and minimum precision). The proposed metric is shown to be better at capturing the overall performance of selective classifiers in the OOD setting.

Strengths

1. The proposed reject option models and evaluation metrics are developed systematically and connected well with summary sections. Found the paper to be well written. 2. The analysis builds upon the prior work [6] which deals with reject option models but not in the OOD setting. In this work, they consider rejecting inputs due to both mis-classification and being OOD. 3. Although the proposed work makes a strong assumption of a known OOD distribution, the analysis is useful to draw insights about the need for a double-score OOD detection method, and also to highlight the shortcomings of existing evaluation metrics. [6] https://www.jmlr.org/papers/volume24/21-0048/21-0048.pdf

Weaknesses

1. The analysis and proposed new metrics (parts of the new metrics like the FPR and precision) depend upon knowledge of the OOD distribution. This is a strong assumption for practical settings. 2. For the proposed double-score OODD method, it seems to me that we need access to OOD data in order to set the hyper-parameter $\mu$ in the combined score $s_r(x) + \mu s_g(x)$. The authors should clarify if this is set based on validation data from a different OOD distribution than the test data. 3. Minor: it is a bit tedious to keep track of all the notations needed for the analysis. **Update after author-reviewer discussions:** \ I have read the authors rebuttal. The paper lacks sufficient discussion on the experiments. Not enough details are provided in the appendix as well. Hence, I decrease my rating to 6.

Questions

1. What is the metric OSCR on line 37? This is defined much later in the paper, please add a reference earlier. 2. Suggestion: “Selection function” seems like a better term than “Selective function”. 3. Line 28: “rejects it to predict” → “rejects it from prediction”. 4. Line 87: in the definition of the expected risk, the expectation should be over $(x, \bar{y}) \sim p(x, \bar{y})$. 5. Some notations are not defined, e.g, the indicator function $c_C(x) = [[ s_C(x) \leq \epsilon_1 ]]$ on line 96 (likewise on line 145). The logical AND symbol $\wedge$ in Eqn (3). 6. In the equation for $R^S(h, c)$ on line 117, the arguments of the loss function seem to be swapped. Should be $\ell(y, h(x))$. 7. In equation (8), it’s not clear to me what the sets $\mathcal{Y}^{\mathcal{X}}$ and $[0, 1]^{\mathcal{X}}$ mean. Please clarify. 8. Statement of Theorem 3 has a minor typo: should be $g(x) = p_O(x) / p_I(x)$. 9. The validation set $\mathcal{T}$ at the start of Section 3 is assumed to have OOD samples from the unknown OOD distribution $p_O(x)$. The OOD samples from the test set follow this same distribution. Is this a reasonable assumption to make for post-hoc tuning? Would not be true in most practical scenarios. 10. Typo on line 230: should be $h^*(x)$. 11. Lines 257, 258: the method should be $D(\mathbb{R})$ and not $E(\mathbb{R})$. 12. For the proposed double-score OODD method denoted by $D(\mathbb{R})$, how is the constant $\mu$ set (e.g. in Table 1)? Is it set by searching for the value that leads to the smallest selective risk (in this case selective error rate) subject to the TPR/FPR constraints? If this is the case, then it could be considered unfair since the OOD distribution in the test dataset is very likely to be different from that of the validation dataset $\mathcal{T}$. Could you clarify this point? 13. In Table 1, why are the AUROC values the same for the methods $A(\infty)$ and $D(\mathbb{R})$? Same question for the AUPR metric. 14. For the experiments in Table 2, can you explain how the methods KNN+MSP and VIM+MSP are combined for the proposed double-score method? As described on lines 202 - 203, one of the OOD methods is used for OOD/ID discrimination and other used for misclassification detection. How do you set the constant $\mu$ used for combining the scores? 15. In Table 2, why is the selective risk at given TPR and FPR the same (or almost the same) for multiple OOD datasets? For example, for CIFAR-10 as in-distribution, all the methods (including the proposed KNN+MSP and VIM+MSP) have the same selective risk across all three OOD datasets.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

Some limitations of the proposed work are mentioned in Section 3.5. Another limitation to include is the fact that the proposed analysis and novel metrics depend on the OOD distribution which is usually unknown. One could use a validation set (as in the paper) with a mix of in-distribution and auxiliary OOD data, but the distribution of auxiliary OOD data in the validation and test sets should be different. Negative societal impacts has not been discussed, but may not be applicable here.

Rebuttal Response

I thank the authors for their response to my comments. > A: Thank you for your valuable feedback. Regarding the assumption about knowing the OOD distribution [...] Thanks for clarifying. It would be important to highlight the fact that practical applications of the concepts from the paper do not require explicit knowledge of the OOD distribution. If I understand the response correctly, then sample access to the OOD distribution is still required. That is a weaker assumption but still might be restrictive in many practical scenarios. > A: Please notice a difference between the reject option model and the evaluation metric [...] > A: The exact formulation of the reject option models for OOD setup, which are described in Sec 2 [...] Thanks for elaborating on this. In combination with the additional elaboration on the purpose of Section 3. This is more clear to me now. > A: We fully agree. Please see Figure 1 in the appendix which is not in the main paper due to lack of space. Thanks for the pointer. It would have been great if the data was visualized. Since the data is 1D this should have been simple (and could have been included in the rebuttal PDF). > A: The reviewer is correct that a detailed description of the experiments is missing from the paper. This is due to a lack of space. However, note that i) the experiments use trained models, scoring functions, evaluation protocol, and data published in [21] and ii) we provide full code in the appendix. > A: Implementation of the compared methods, pre-trained models, the data, and their splits into training/validation/test sets are all adopted from the OpenOOD [21]. Unfortunately, I am not satisfied with these responses. At the very least, the appendix should have detailed the experimental setup, even if it is mostly shared from previous works. Moreover, my comment on MNIST vs CIFAR-10 as well as the proposal to consider more real-life shifts was not addressed by the authors. --- I have increased my score in light of the clarification around section 3. I still have some reservations wrt the OOD distribution assumption and especially the experimental evaluation which prevents me from further raising my score.

Rebuttal acknowledgement

Thank you for the clarifications. I have read through all the reviews and rebuttals, and most of my questions/concerns have been resolved. It would help to address these points clearly in the revised paper (e.g., the fact that the practical use of the proposed work does not require exact OOD knowledge).