Adaptive Uncertainty Estimation via High-Dimensional Testing on Latent Representations

Summary

The paper presents a new framework for uncertainty estimation in baysian neural networks. The core contribution is formulating uncertainty estimation as multiple high-dimensional hypothesis testing problem and deriving the test statistics necessary. The paper then present multiple empirical results, showing good performance in well established benchmarks and a excellent ablation study of relevant hyperparameters of their proposed method.

Strengths

Originality: The core idea of the paper of formulating uncertainty estimation as hypothesis testing seems novel. The paper essentially combines already established methods, BNNs with hypothesis testing (both which are well know), into a new exiting direction for this line of work. Quality: The paper is of high quality, both in outlining the method and their experimental section. For example, it is nice to see standard deviations added for nearly all results and a lot of relevant baselines to compare against. The inclusion of DRD datset, to test the method on a more "real-life" dataset is also a welcome addition and really shows the that the methods does not only work on benchmark datasets. Clarity: The paper is well written and easy to follow. Significance: The paper does open a new sub-direction within uncertainty estimation, regarding the use of hypothesis testing. Only time will tell how influential this will be. I am not truly convinced that the ARHT testing size is the correct way to go as it still involves an assumption of sub-Gaussuanity on its input. The core strength of the paper is its section 5 regarding the ablation studies. The framework presented in the paper includes a lot of "moving parts" and therefore without a thoroughly investigation on which parts of this many layered method, it would be impossible to say which had a significant impact on the performance of the framework. The ablation study does a good job highlighting how each component feeds into the framework. Also, thanks to the authors for providing anonymised code, helped understand parts of the paper.

Weaknesses

In L56-58 the authors claims that "We formulate uncertainty estimation as a multiple high-dimensional hypothesis testing problem, and propose a Bayesian deep learning module to address better the aleatoric and epistemic uncertainties when learning feature distributions.". While I agree with the first part of the sentence, I would say the second part is not a contribution as too my knowledge the BNN framework used in the paper is not novel, which there is nothing wrong with, but should not be listed as a contribution. In that regard I think the authors could make it more explicit that the first part of their framework is not unique in any sense, and therefore could be any BNN. Regarding results, similarly the authors should be more honest about their results. L206-206 state that their method is "superior", however if accounting for standard deviations the results only seems significant in a single case. Thus overall, authors should adjust the claims in their paper to better reflect the work presented. A small correction/question: From L130-139, the authors provide a bunch of equations, where the choice of parenthesises seems very arbitrarily. Sometimes {} is used, sometimes [] is used. sometimes () is used. It is confusing. Additionally in L139 there seems to be missing a "/" between what I assume is the nominator and denominator in a fraction.

Questions

* Have the authors any thoughts on the influence of n_1, e.g. the size of the training set? For all datasets tested in the paper this is fixed, but the number of samples in the training distribution compared to the test distribution seems to be important for this method to work. * Regarding Figure 5: Have the authors tried out this kind of ablation studies on other dataset combinations to see if they experience similar pattern? Mnist-FMnist is one of the easier cases for OOD and I wonder if this has something to do with the observed robustness of the method. * Regarding Figure 5: The results from varying n_2 is strange to me. I would assume as we sample more test datapoints we get more certain about the test distribution making it easier to distinguish for the test distribution, however this does not seem to be the case. Does the authors have an explanation for this?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

As the authors state in L300-302, BNNs are still hard to scale which seems to be the main limitation this line of work. That said, assuming that in future this gets solved this method seems very easy to apply. Another limitation of the method, seems to be that it is limited to OOD, or at least tasks where distributions are compared. OOD is an important tasks, however just estimating the predictive uncertainty of a single sample is similarly important, which the framework does not seem to support.

Summary

This paper proposes a framework to detect out-of-distribution (OOD) data via high-dimensional testing on latent representations. The proposed framework consists of: - a Bayesian Neural Network that, for any input, can produce an ensemble of latent presentations by sampling the posterior of the weights - a hypothesis testing procedure that computes the adaptable regularized Hotelling's $T^2$ score (ARHT) as a measure of uncertainty, and classifies the input as OOD when the ARHT score is larger than a threshold calibrated by the Benjamini-Hochberg (BH) procedure The paper demonstrates the superior performance of the proposed framework on standard OOD detection benchmarks and a medical image dataset.

Strengths

- The idea of using Bayesian NN to generate an ensemble of latent representations, and then leveraging the ARHT score as a measure of uncertainty is novel, interesting, and useful for many practical applications - The paper is generally well written and easy to follow.

Weaknesses

- A major weakness of this paper is that, for readers who do not have a statistical background, the ARHT score, i.e., equations (1)-(4), appear to come from nowhere. Particularly, I think the paper will be much easier to appreciate if comments and explanation could be made about: - What is the intuitive explanation of the Hotelling's $T^2$ test statistic $T$? Why should we believe it is a good metric for separating in-distribution and OOD samples? - What is the motivation of loading a scaled identity matrix to $T$? In the related work section (which appears at the end of the paper) you seem to state that was due to the potential singularity of the matrix $S_n$. I believe you should state this earlier when you present the equation (2) and (3). - Why the ARHT score $T(\lambda)$ satisfies a Gaussian distribution $\mathcal{N}(0,1)$? This claim seems to come from reference [24]. I believe you need to briefly explain this to the reader. - Section 3.4 describes the B-H procedure without intuitive explanation and detailed derivation. I think more contents are need to explain this, at least in the supplementary materials. - There are many typos in the manuscript. The authors should carefully review the grammar if the paper was accepted.

Questions

- The paper says that $T(\lambda) \sim \mathcal{N}(0,1)$, but looking at Figure 4, the scores do not appear to be Gaussian, and the scale of the scores grows to several thousands. Is my understanding about Fig. 4 wrong, or there is some mistake in plotting the figure? - To computer $RHT(\lambda)$ in equation (3), one needs to invert the matrix $S_n + \lambda I_p$, which has size $p$ that is the dimension of the latent representation. What if $p$ grows to, say one million? Would it still be possible to compute the score? Also the paper does not seem to clearly state the runtime of the proposed algorithm.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

N/A

Summary

The paper proposes BNN-ARHT, which introduces a uncertainty estimation framework that uses high dimensional hypothesis testing in the feature space of a network. The key idea is to use ARHT to determine in vs outliers in a feature space, and so it is generic and broadly applicable to any kind of task and network type trained with supervised or unsupervised objectives. In particular, the paper relies on a Bayesian encoder to compute the ARHT on train and test data, that is trained with Stochastic Variational Inference (SVI). The testing statistic is the Hotelling’s $T^2$ test statistic, of which the adaptable and regularized version is used. The threshold $\lambda$ is tuned from a pre-defined set for each sample, and for each task separately which is argued to produce the best results

Strengths

* ARHT seems to be a interesting and novel statistic from the uncertainty estimation point of view. * An appealing aspect of the proposed approach is that it can work in arbitrary feature spaces, which vastly expands its applicability. This is also relatively under studied as compared to most approaches that compute uncertainty in the output space. * Predictive performance on the networks tested is preserved, while retaining benefits of OOD detection. Good ablation studies are performed to understand the characteristics of the method. * The finding that the method can be used with a encoder trained with supervised or unsupervised architecture is very interesting, and raises several intriguing questions. This can be studied in more detail — perhaps a related work is "Predictive Inference with Feature Conformal Prediction” from ICLR 2023. https://arxiv.org/abs/2210.00173

Weaknesses

* The paper only compares with few uncertainty estimation methods for the OOD detection setup — whereas i think it should compare more comprehensively with the wide range of state of the art techniques that have achieved high performance on all the benchmarks considered here. * The lack of more realistic OOD benchmarks is a limitation. The experiments are only done on CIFAR10/MNIST with simple networks (ResNet18) whereas it is essential to validate these interesting claims on more realistic benchmarks. * The generality of the approach needs to be rigorously evaluated — UQ for regression, and tasks other than classification can be evaluated and compared against existing methods. * On UQ -- why is OOD the only application considered? OOD benchmarks are often artificial and can have a lot of artifacts which can aid in detection (See for e.g. Semantically Coherent Out-of-Distribution Detection, ICCV '21). Tasks like Bayesian function optimization can shed more light on the quality of uncertainties, and its performance on regression. * Scalability needs to be addressed — a big limitation is the scalability of the proposed approach to bigger and more realistic datasets and networks. Even for ResNet18 considered here, the only a single layer is replaced with its BNN equivalent. it’s unclear why the last layer is optimal, or how this choice can be made in other settings. Further, the threshold needs to be identified each time for each task, which is a computational burden, that other competing methods do not have.

Questions

My main concerns are regarding the lack of more comprehensive evaluation for OOD detection. Using better baselines, large scale benchmarks and more modern architectures.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Yes

Summary

This paper proposes an OOD detection procedure by applying adaptable regularized Hotelling’s T-square (ARHT) test [24] on the feature representation of learned BNN networks. Authors introduced the application o ARHT on BNN encoder, and proposed a procedure to adaptively calibrate detection threshold based on Benjamini–Hochberg (BH) procedure. On a suite of low-dimensional image experiments (CIFAR, MNIST, OMNIGLOT, SVHN) and a medical image benchmark (DRD) and using small architectures (e.g., LeNet, AlexNet, ResNet18), authors illustrates that the proposed method clearly outperforms previous methods in terms of OOD AUC. Thorough ablation study is conducted.

Strengths

* An novel approach to OOD detection leveraging multivariate tests and associated calibration procedures. * Authors conducted thorough experiment on academic benchmarks to show the advantage of the approach.

Weaknesses

* Some description of previous literature may be incorrect. For example, in line 43-48, authors mentions that (1) BNNs perform poorly when the dimension of the output is high. (2) BNN are limited to classification problems. I am not sure if either of these are true for modern BNNs. For example, [rank-1 BNN](https://arxiv.org/pdf/2005.07186.pdf) outperforms their deterministic counterparts on ImageNet, and regression is a rather standard BNN task (e.g., conducted on UCI benchmarks). * The experiments are done on rather simplistic architectures and benchmarks. Therefore the generalization of this approach (either in terms of quality and computational feasibility) to more realistic data setting (e.g., ImageNet) and nontrival modern large models is less clear. * I find that the point "our framework is sensitive to the quality of the encoder" (line 260-266) to be rather important. Three suggestions regarding this point: * this seems to contradict the paper's claim that ARHT is superior to traditional BNN whose "performances heavily rely on the feature encoder and are poor when the features are of poor quality" (line 48). As it seems ARHT suffer a similar issue but to a lesser extent. I recommend adjust the descriptions in 48-50 correspondingly so it is not misleading (maybe refocus (3) in terms of sample efficiency). * It would be interesting to have a more thorough investigation on the relationship between encoder quality v.s. OOD detection performance, where encoder quality can be quantified in terms of standard representation learning metric such as linear probing accuracy. This can be done via a scaling study of different sample sizes and / or architectures. I think a study like this can possibly show ARHT performance is associated with encoder quality, but can achieve a stronger OOD performance when compared to other method under the same accuracy. * Please consider discussing this in the limitation section.

Questions

See Weakness.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

See Weakness.

response

Dear authors, thank you for the rebuttal. This answers some of my questions, but many still remain. First, i still think scaling is a big weakness of the proposed approach. While it is encouraging to see it being scaled to Resnet-50, i find the performance quite surprising. Weaker models on Cifar-10/SVHN have performed far superior to the numbers reported here -- can the authors explain why? For e.g. Deep Ensembles has an AUROC of only 65 here, where as previously published work has shown even with Resnet-18 to have AUROC of 90+ even with a 3-model ensemble. (See table 3, http://proceedings.mlr.press/v119/van-amersfoort20a/van-amersfoort20a.pdf). Is this because of the scoring function used? Even a simple MSP with resnet-18 should give high AUROC on CIFAR-10/SVHN, which makes the justification of the proposed approach weaker. Next, by scalability of datasets, I meant the in distribution datasets (such as ImageNet) and not necessarily OOD sets like TinyImageNet -- though this is still good evidence to have in support of your technique. Next, by regression tasks, I was primarily talking about uncertainty in regression settings which is much broader (such as function optimization) but it is encouraging to see the synthetic experiment nevertheless.

Answer to authors

I thank the authors for the response to my questions. I am glad that the authors are willing to change some of the language in their paper to better reflect the paper contributions. Additionally, I am happy with the response to my questions regarding some of the hyperparameters of the method. I hope some of this will be included in the camera-ready version or appendix in the future. Based on the response from the authors and the other reviews, I will be keeping my score for now.