Efficient Uncertainty Quantification and Reduction for Over-Parameterized Neural Networks

Uncertainty quantification (UQ) is important for reliability assessment and enhancement of machine learning models. In deep learning, uncertainties arise not only from data, but also from the training procedure that often injects substantial noises and biases. These hinder the attainment of statistical guarantees and, moreover, impose computational challenges on UQ due to the need for repeated network retraining. Building upon the recent neural tangent kernel theory, we create statistically guaranteed schemes to principally \emph{characterize}, and \emph{remove}, the uncertainty of over-parameterized neural networks with very low computation effort. In particular, our approach, based on what we call a procedural-noise-correcting (PNC) predictor, removes the procedural uncertainty by using only \emph{one} auxiliary network that is trained on a suitably labeled dataset, instead of many retrained networks employed in deep ensembles. Moreover, by combining our PNC predictor with suitable light-computation resampling methods, we build several approaches to construct asymptotically exact-coverage confidence intervals using as low as four trained networks without additional overheads.

Paper

References (100)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer nEzW7/10 · confidence 3/52023-06-28

Summary

This paper proposes an epistemic uncertainty assessment framework which comes with statistical coverage guarantees and low computation costs for over-parametrized neural networks. This approach seeks to remove procedural uncertainty by using one auxiliary network. In addition, approaches are provided to construct confidence intervals using a small number of retrained networks on different problem settings.

Strengths

The authors introduce a Procedural-Noise-Correcting (PNC) predictor, which uses one auxiliary network trained on the data but where all labels set to zero (instead of +-1). This mimicks the variability coming from the training procedure leveraging asymptotic results from NTK theory. The idea is simple and intuitive and appears to work on some toy experiments.

Weaknesses

Experiments are only provided on low-dimensional toy problems. In addition, for the coverage studies, only 40 samples are used which results in relatively large binomial errors on the reported coverage properties. This issue isn't discussed at all. It would be better to both increase the number of samples and to report the results in a way that respects the uncertainty due to finite samples.

Questions

How wide is wide enough that the NTK theory used is approximately valid? Have you looked into studying more realistic problems (even if still only simulated)?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Some discussion should be provided on limitations. For example, how large does the NN need to be?

Reviewer bPtq6/10 · confidence 3/52023-06-29

Summary

The paper focuses on uncertainty quantification, specifically on estimating the statistical range of predictions made by a deep neural network (DNN). This is achieved through the use of a novel DNN called Procedural Noise Correcting (PNC), which is capable of estimating the variability inherent in the training process, particularly the variability arising from different initializations of the DNN. In order to obtain a more accurate estimate of the idealized DNN, denoted as h^*, the authors employ the Neural Tangent Kernel (NTK) and train the PNC DNN. To construct the confidence interval for the PNC, they utilize either batching or bootstrap methods.

Strengths

In this paper, the authors introduce a remarkable deep neural network (DNN) named PNC, which effectively addresses uncertainty quantification. The presentation of the paper is well written, although there are a few missing notations. However, the supplementary materials provided by the authors greatly enhance the overall completeness of the paper. The problem tackled by the authors is intriguing, and to the best of my knowledge, I have not come across any other papers that address this particular issue using a similar approach.

Weaknesses

There are several instances of missing notations in the paper, which make it challenging to comprehend. I'm curious about the authors' decision to only consider the random initialization of network parameters when discussing procedural uncertainty. Why did they not take into account random batches or the non-deterministic backpropagation? Algorithm 1 is also difficult for me to understand. If only \bar(s) is predicted, how can h_n*(x) be obtained? Is it necessary to apply the NTK? I did not see it explicitly mentioned in algorithm 1. Regarding the experimental protocol, I believe it could benefit from a more robust approach. It might be valuable to compare the proposed method to other approaches, such as: [1] Blundell, Charles, et al. "Weight uncertainty in neural network." International conference on machine learning. PMLR, 2015. [2] Maddox, W. J., Izmailov, P., Garipov, T., Vetrov, D. P., & Wilson, A. G. (2019). A simple baseline for bayesian uncertainty in deep learning. Advances in neural information processing systems, 32. [3] He, Bobby, Balaji Lakshminarayanan, and Yee Whye Teh. "Bayesian deep ensembles via the neural tangent kernel." Advances in neural information processing systems 33 (2020): 1010-1022. Additionally, I find it intriguing to include experiments on additional datasets such as MNIST, CIFAR-10, and CIFAR-100, not limited to regression tasks but also for classification purposes. It would provide valuable insights into how the proposed DNN performs in the presence of epistemic uncertainty. Why not evaluate its behavior by applying out-of-distribution (OOD) data? The authors highlight that PNC involves minimal computation; however, they only apply their technique to relatively simple datasets. This raises questions about whether it truly qualifies as a low-budget technique.

Questions

Please see Weaknesses

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

The authors' paper lacks emphasis on the societal impact of their work. Typically, I wouldn't expect authors in the field of Uncertainty Quantification to delve into this aspect. However, I do feel that the claims made in the paper are overly assertive. For instance, on line 29, the paper mentions general uncertainty quantification, yet it only focuses on regression tasks. Additionally, the claim of working with Overparametrized Neural Networks seems exaggerated, considering that the DNN employed in the study consists of just two fully connected layers. It would be advisable for the authors to exercise caution when making such claims.

Reviewer cgMr6/10 · confidence 3/52023-06-30

Summary

The authors focus on the task of uncertainty quantification for neural networks. They contribute (i) a procedure to remove procedural uncertainty, an uncertainty that arises due to randomness in the training procedure, and (ii) an approach to cheaply construct confidence intervals with asymptotic coverage guarantees. The proposal is evaluated on two small synthetic experiments.

Strengths

- The proposed approach offers a cheap (compared to ensemble models) and principled approach to remove procedural uncertainty and construct confidence intervals. - The paper is well written.

Weaknesses

The main weakness of the paper is its empirical evaluation, which consists of a single simple data set, a sum of sine functions, evaluated with small data sets (up to 1024) in very low dimensional spaces (up to eight dimensions). Any kind of scalability guarantee is not given/evaluated, which is concerning, especially since the theory assumes access to the full training data (l118). A second synthetic function (sum of exponentials and squared covariates) is evaluated in the appendix, again without too many details. ### Minor - Throughout the paper, the term _extremely_ is used repeatedly (extremely fast, extremely time-consuming, extremely few) instead of providing proper quantitative numbers (if only in the order of magnitudes, or relative to other approaches). - Prop 3.4 $\pi_n$ is only defined in the appendix - When submitting to NeurIPS, please follow the NeurIPS style guide, i.e., place your captions above tables, no vertical lines in tables, etc. - The regularizing $\lambda$ is fixed to $\lambda \equiv 0.1^{10}$, i.e., essentially zero? Why keep it at all?

Questions

- The evaluation is limited to a single function in a very low-dimensional setting. Can the authors provide a further experimental evaluation with respect to the scalability of the proposed approach? - Wrt the paragraph around l174-l193. I am not sure I understand this. $\bar s \equiv 0.2$, or $\bar s \equiv 0$, gives us a constant target, that is independent of the input, i.e., $\hat \phi_{n, \theta^b}'(\cdot)$ should learn to be constant, giving us $\bar \phi_{n,\theta^b} \equiv 0.2$, i.e., step 2 of Algorithm 1 becomes irrelevant. Which part am I misunderstanding?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Discussion on limitations and broader impact are missing.

Reviewer VEbh6/10 · confidence 3/52023-07-03

Summary

In the paper, authors present a new approach to quantify and mitigate a specific aspect of epistemic uncertainty in model predictions. They identify and quantify "procedural variability," a type of epistemic uncertainty that arises from noise in the training process. Based on the Neural Tangent Kernel (NTK) theory, the authors introduce a "procedure-noise-correction" method. They offer theoretical support and experimental evidence to justify their approach.

Strengths

Generally, the majority of research papers propose methods that quantify the overall epistemic uncertainty, but a few discuss the estimation and evaluation of its different components. The process of splitting epistemic uncertainty into different parts, while not a novel concept, remains significantly under-studied. Therefore, the problem addressed by this paper is promising and brings potential value for further studies. The strengths of this paper are: 1) The authors built a mathematical framework for their proposed method. 2) The authors' approach to quantify and remove procedural variability is indeed novel and opens new perspectives for future works. 3) Combined with the PNC-Enhanced batching, the presented approach seems practical, and practitioners in the field may find it beneficial. 4) The supplementary material is verbose and resembles a textbook, providing a comprehensive guide to the details of derivations, proofs, and certain theoretical aspects like the NTK.

Weaknesses

There are some aspects of this paper I consider as weaknesses: 1. Narrative of the Paper: Despite its overall good organization and clarity, Section 3 in the main document is challenging to understand. Reorganizing this section could enhance the paper's readability. Some propositions could be moved to the Appendix, freeing up space for the experimental section (see Weakness 2). Adding illustrative elements, such as a schematic diagram of the dataset construction and overall architecture, could further improve the readers' understanding. 2. Experiments: The set of experiments presented appear overly simplified and unrepresentative. Even though the multi-layer perceptron (MLP) models with two layers and varying unit numbers technically qualify as over-parameterized for d=1,2,4,8, the example is overly simplistic. The only experiment is with a sinusoidal signal form, and a y-label variance of 0.001 isn't explained or justified. The paper could be enriched by the inclusion of generally accepted benchmarks, such as regression benchmarks containing UCI datasets[1]. Moreover, it's clearly seen from your results that an ensemble of 5 models substantially outperforms an ensemble of 2 models, and training these ensembles isn't overly computationally demanding considering the complexity of data and models. Hence, comparing with larger ensembles, say 50 or 100 members, could provide more insightful results. 3. The initial part of the paper seems to promote the enhancement of **uncertainty quantification** by disentangling epistemic uncertainty into different components, but the actual evaluation of this uncertainty is lacking. The experiment appears to improve the **optimization process** rather than the quantification of uncertainty. An experimental suggestion would be to select a measure for epistemic uncertainty and apply it to both in-distribution (validation split) and out-of-distribution data. Computing the uncertainty measure for each data point across independently trained models several times and comparing the resulting ROC-AUC with the one obtained through your method could demonstrate whether the elimination of procedural variability truly improves the estimates of epistemic uncertainty. 4. There is no code available, so I can not check the reproduction of the experiments. [1] Gal Y., Ghahramani Z. Dropout as a bayesian approximation: Representing model uncertainty in deep learning //international conference on machine learning. – PMLR, 2016. – С. 1050-1059.

Questions

Can the method be somehow adapted for classification? Edit: I would like to thank the authors for their answers during rebuttal period.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

As I can see, this paper has no potential negative societal impact.

Reviewer nEzW2023-08-13

I think you missed the point about the coverage estimates. Any finite number of repetitions used to estimate the coverage will incur a binomial error on the coverage estimate. In the paper you ignore this and have examples that are 1 sigma from the desired asymptotic value that are labeled as undercoverage. I would suggest representing that better and doing more repetitions to reduce the uncertainty. Nevertheless based on the other replies I've raised by score by one.

Authorsrebuttal2023-08-14

We greatly appreciate the reviewer for increasing our score, and we also apologize for not adequately addressing the issue on coverage estimation accuracy. We now see the point of the reviewer. In the final version, we will 1) increase the number of experimental repetitions from 40 to at least in the hundreds. 2) report the confidence interval of the coverage by using the binomial or sample proportion confidence interval. Both of these would serve to better represent the comparison results.

Reviewer VEbh2023-08-15

I would like to thank the authors for their detailed answers! I am quite confused with the answer to Q2. In particular: ``epistemic uncertainty is typically used on in-distribution data, ... detecting out-of-distribution data generally requires aleatoric uncertainty''. Can you please elaborate on what precisely you mean? If aleatoric uncertainty is captured by $\pi_{Y \mid X}$, does not it mean that aleatoric uncertainty makes sense only for that $X$ which came from the in-distribution? Otherwise not clear what should be the distribution over $Y$ for out-of-distribution $X$.

Authorsrebuttal2023-08-16

We appreciate the reviewer for your reading and reply. We apologize for not adequately explaining the comment on the out-of-distribution detection. Let us elaborate on this point a bit more. There are two different scenarios for detecting out-of-distribution data, as follows: 1) The goal is to check whether the joint distributions are identical, i.e., whether $(X_{test}, Y_{test})$ is from the same distribution as $(X_{train}, Y_{train})$. To this end, we can estimate the aleatoric uncertainty $\pi_{Y \mid X}$ and check whether $Y_{test}$ falls into the highest density region of $\pi_{Y \mid X_{test}}$ [1]. If the answer is no, then with high probability, $Y_{test}|X_{test}$ is out-of-distribution, and thus $(X_{test}, Y_{test})$ is also out-of-distribution. This approach considers the conditional distributions of Y given X. This is the scenario we referred to in the rebuttal, which is indeed related to our problem setting. In particular, our problem aims to quantify the uncertainty in the obtained model $\hat{h}(x)$ against the target best model $h^*(x)$ (with a fixed $x$). We do so by providing a confidence interval around $h^*(X_{test})$ (not $Y_{test}$), because we focus on the quantification of epistemic uncertainty. On the other hand, $Y_{test}$ typically contains noise in regression tasks (i.e., the aleatoric uncertainty is not zero), and to carry out the out-of-distribution detection scheme, the construction of the highest density region needs to consider the aleatoric uncertainty. 2) The reviewer probably means the second scenario that has been widely studied in classification: The goal is to check whether the marginal distributions are identical, i.e., whether $X_{test}$ is from the same distribution as $X_{train}$ (typically given some conditions, such as for a certain class). To our knowledge, this task generally requires estimating the distribution of $X_{train}$ or its features (extracted from the neural network), and then checking whether $X_{test}$ fits. For instance, [2,3] use information from the likelihood function based on the estimated density function of $X_{train}$ from generative models, [4,5] check the energy score or softmax score from the distribution of the final layer's logits, [6] aggregates the p-values from the distribution of intermediate layers’ features. These works are related to quantifying the distribution of $X_{train}$ or its features (logits can also be viewed as condensed features), while our work has a different focus: The epistemic uncertainty is regarding the response $\hat{h}(x)$ while $x$ is a fixed test point (not random), and the randomness is from the training data and training procedure (different in-distribution training data and training procedures lead to different responses $\hat{h}(x)$). References: [1] Hyndman, Rob J. "Computing and graphing highest density regions." The American Statistician 50.2 (1996): 120-126. [2] Nalisnick, Eric, et al. "Do Deep Generative Models Know What They Don't Know?." International Conference on Learning Representations. 2019. [3] Grathwohl, Will, et al. "Your classifier is secretly an energy based model and you should treat it like one." International Conference on Learning Representations. 2020. [4] Liu, Weitang, et al. "Energy-based out-of-distribution detection." Advances in neural information processing systems 33 (2020): 21464-21475. [5] Hendrycks, Dan, and Kevin Gimpel. "A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks." International Conference on Learning Representations. 2017. [6] Haroush, Matan, et al. "A Statistical Framework for Efficient Out of Distribution Detection in Deep Neural Networks." International Conference on Learning Representations. 2022.

Reviewer VEbh2023-08-18

Dear authors, Thank you for your answers. I think you resolved my confusion. Let me explain. I have some experience in uncertainty quantification, especially epistemic uncertainty quantification, and out-of-distribution detection in classification. In the problems I was addressing, we typically have some unrelated objects as out-of-distribution samples. Imagine that the source task was to classify cats and dogs, while during the test there appeared an image of a person's face. This is why I asked how $\pi(Y\mid X)$ can even be related for out-of-distribution samples (in my previous example, what should be the distribution over 'cat' and 'dog' labels given an image of a person's face)? Now I realize that you rather address ``outliers'', but not totally unrelated objects as out-of-distribution. I think some clarification of this in the main text may help people (with similar previous experiences as mine) to understand the paper better. Based on the discussion, I would like to raise my evaluation score.

Authorsrebuttal2023-08-19

We greatly appreciate the reviewer for your reply, including the kind sharing of your experience, and also for increasing our score. The discussion would also help us phrase our paper better. We would make sure to clarify our relation with out-of-distribution detection in the main body of the final version of this paper, as kindly suggested by the reviewer.

Reviewer bPtq2023-08-15

Dear Authors, I extend my gratitude for your response. After revisiting the paper alongside your answer, I find that the content has become notably clearer. Regarding your responses to **points 1-4**, I find myself in agreement with the explanations provided by the authors. As for **point 5**, I hold a differing perspective from what the authors refer to as a 'Deep' Neural network. While I may not entirely align with the terminology used, acknowledging the variation in nomenclature within the field, I am open to conceding this point. In light of the clarifications and persuasive arguments presented, **I am pleased to convey my decision to revise my initial review.**

Authorsrebuttal2023-08-16

We greatly appreciate the reviewer for your support and for increasing our score. We are also glad to hear that our response helped make the contents notably clearer. Regarding the terminology, we will make sure to clarify it in the final version to avoid overclaims in the paper as kindly suggested by the reviewer.

Authorsrebuttal2023-08-16

Thank you very much. We appreciate the reviewer for your reading and reply.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC