Proximity-Informed Calibration for Deep Neural Networks

Confidence calibration is central to providing accurate and interpretable uncertainty estimates, especially under safety-critical scenarios. However, we find that existing calibration algorithms often overlook the issue of *proximity bias*, a phenomenon where models tend to be more overconfident in low proximity data (i.e., data lying in the sparse region of the data distribution) compared to high proximity samples, and thus suffer from inconsistent miscalibration across different proximity samples. We examine the problem over 504 pretrained ImageNet models and observe that: 1) Proximity bias exists across a wide variety of model architectures and sizes; 2) Transformer-based models are relatively more susceptible to proximity bias than CNN-based models; 3) Proximity bias persists even after performing popular calibration algorithms like temperature scaling; 4) Models tend to overfit more heavily on low proximity samples than on high proximity samples. Motivated by the empirical findings, we propose ProCal, a plug-and-play algorithm with a theoretical guarantee to adjust sample confidence based on proximity. To further quantify the effectiveness of calibration algorithms in mitigating proximity bias, we introduce proximity-informed expected calibration error (PIECE) with theoretical analysis. We show that ProCal is effective in addressing proximity bias and improving calibration on balanced, long-tail, and distribution-shift settings under four metrics over various model architectures. We believe our findings on proximity bias will guide the development of *fairer and better-calibrated* models, contributing to the broader pursuit of trustworthy AI. Our code is available at: https://github.com/MiaoXiong2320/ProximityBias-Calibration.

Paper

Similar papers

Peer review

Reviewer 4HP96/10 · confidence 3/52023-06-29

Summary

This paper quantifies and proposes a mitigation for a phenomenon in DNN training, where more 'unusual' examples (here defined as having a higher average distance to its K=10 nearest neighbors) are generally more miscalibrated across a range of models and tasks. The authors propose a new proximity-aware calibration metric, PIECE, and demonstrate that it can capture calibration issues that are not necessarily captured by the standard ECE metric (and, in fact, always P>= ECE). Additionally, the authors propose a mitigation, ProCal. This mitigation is presented in two variations, one for continuous and one for discrete confidence. These work by adjusting the uncalibrated probability score based on the model's miscalibration on examples with that average distance.

Strengths

The paper is very clearly written, with a logical flow and good explanations of all steps taken. I particularly enjoyed the motivation via experiments on existing models leading to theory-driven proposals leading to further experimental verification. I also appreciated the through ablation study and the extra attention given to OOD examples. In addition, the original miscalibration problem that the paper brings up seems to me to be important and relevant, and is well-motivated. As I am not an expert in miscalibration mitigation techniques I cannot fully comment on the novelty aspect of the work, though I have not seen this 'atypicality' concern brought up explicitly before; it also dovetails neatly with other research in atypical examples. The proposed metric and mitigations, while very simple, are logical and effective, which I hope will lead to their adoption.

Weaknesses

I felt that the 'atypical' (high $D(X)$) examples could have been better characterized. In particular, the authors relate these examples to underrepresented categories in datasets (eg, Black people in health datasets), but this is not necessarily so. It does not seem incredible to have a scenario where a data is divided into two clusters, where the smallest cluster is nevertheless very tight, and so has a smaller average $D(X)$. It is not really clear what the ProCAL method refers to. It seems like it mostly splits into two methods each of which have their own name. Maybe it would be better to call them ProCAL-C and ProCAL-D (for continuous and discrete)? The analysis of ProCAL effectiveness seems a little sparse in claiming "our method consistently improves the calibration"... My interpretation of the data would be that ProCAL is very helpful in conjunction with raw confidence as well as some of the other methods with higher PIECE scores (TS variants), but not as effective for methods such as IR and MIR, where it seems to hurt as much as help. I don't think that this disqualifies the paper, but a more thorough explanation and analysis of this would be appropriate. (minor) in section 3.1, what is the sensitivity of two points having the 'same' confidence? (minor) It is not clear why the definition of D(X) contains an exponent, rather than the simple average of the ten smallest distances.

Questions

Please see the weakness section above. In addition: * For cases where the test distribution doesn't match the train distribution, how much are we losing by using the test data to estimate values of D (versus measuring the distance of a test sample point to the closest 10 training points)? It would seem that the training distribution is actually the important one for estimating atypicality. * How does the Bias Index changed if only $B_H$ is used for sampling points and $B_L$ is used for matching, and vice versa? What if the intermediate groups are used? * Is it possible to characterize the $B_H$ and $B_L$ groups? Is there a large difference of average confidence between the two, and how well do their confidences intersect? * Why is ProCAL less effective in conjunction with IR and MIR?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

The limitations are adequately addressed.

Reviewer 4gc87/10 · confidence 4/52023-07-05

Summary

This paper studies the prevalence of proximity bias in calibration, i.e. the rate of miscalibration on samples that are far away from their nearest neighbors in the data ("low proximity"). The authors empirically show that this type of miscalibration is present across many models, and propose a new post-training calibration procedure for mitigating it. Their approach shows significant empirical improvements over standard post-training calibration approaches.

Strengths

- **Originality:** The empirical investigation and proposed methodology in the paper is quite novel, as I am not aware of prior work that has studied this type of bias in calibration (although others have studied subgroup calibration). - **Quality:** The claims in the paper are technically sound, and the experiments exploring proximity bias are extensive (more than 500 pretrained models considered, and various calibration baselines). - **Clarity:** Overall, the paper is well-written and well-organized, with motivating experiments and intuitive definitions. However, I believe the presentation of the calibration algorithm in the paper could be improved (detailed further in weaknesses). - **Significance:** The idea of low proximity samples introduced and analyzed by the authors seems quite significant, as these samples can correspond to underrepresented populations in the data.

Weaknesses

- **Algorithm Details:** The weakest part of the paper in my view is the lack of detail in Section 5.1. This part of the paper would be significantly improved by including something akin to the pseudo-code algorithm in the appendix. There are several questions that arise when reading this part: what does one do after estimating the posterior probability conditional on prediction and proximity (in the algorithm I can see that this is just the output on the test point)? Do we compute proximity only with respect to the test data? What type of KDE is used (i.e. kernel, bandwidth, etc.)? - **Implementation Details:** In addition to algorithm details, some parts of the experimental setup could also be made clearer. How are ECE and ACE computed (i.e. binning scheme)? Do you set aside calibration data for the scaling methods (TS, ETS, PTS, PTSK) in addition to the set aside data for ProCal?

Questions

The main questions I have are detailed in weaknesses above.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

4 excellent

Limitations

The authors appropriately discuss limitations, but it might be helpful to include some bits from the appendix (particularly regarding efficiency) in the main body.

Reviewer 1D757/10 · confidence 5/52023-07-06

Summary

The article focuses on the problem of uncertainty quantification in classification. Calibration provides some guarantees on the estimated class probabilities on average. However, subgroups can still be miscalibrated. The article first aims to characterize these subgroup miscalibrations through proximity levels of the samples. It claims that a classifier, even calibrated, tends to be underconfident on high-proximity samples and overconfident on low-proximity samples. To measure this effect, it defines a proximity-informed ECE. Then, it proposes a recalibration framework based on this proximity-informed measure. Finally, it benchmarks the proposed method on numerous datasets and models.

Strengths

* The problem is well presented and motivated. Figure 1 is pedagogical and helps the comprehension of the problem. * The idea of characterizing subgroup miscalibrations through proximity is interesting, and refining uncertainty estimates is a good direction. * The proposed framework is versatile: it can work as a stand-alone or combined with standard calibration techniques. It provides two versions: binning-based and continuous. * The experimental study is substantial. * Datasets are large-scale, numerous, and multimodal: ImageNet, Yahoo-Topics, iNaturalist, ImageNet-LT, MultiNLI, ImageNet-C. * The article studies numerous models, e.g. 504 pre-trained models on ImageNet. * It compares many standard calibration methods, both scaling-based and histogram-based: temperature scaling, ensemble temperature scaling, parameterized temperature scaling, histogram binning, isotonic regression, and multi-isotonic regression. * The experimental study provides substantial evidence. * It reveals proximity bias in most of the 504 pre-trained networks on ImageNet (72% according to a Wilcoxon rank-sum test). * The proposed method consistently improves over standard calibration methods. * The time overhead of the method is small, with an increase of 1.17% in inference runtime. * Completeness of the study: It reveals the proximity bias, proposes a metric to measure it, a recalibration procedure to address it, and substantial experiments showing consistent improvements.

Weaknesses

No major weaknesses.

Questions

* In equation (5) on the confidence score adjustments, why a weight hyperparameter $\lambda \in (0, 1]$ is needed? Don't we want to take $\lambda=1$ to minimize PIECE? The paper talks quickly about regularization. What is the tradeoff? Are results wrong with $\lambda$ too close to 1? Do you take $\lambda=0.5$ everywhere in the experiments? How to choose it? * Concerning the time overhead introduced by the recalibration methods, isn't it weird that Bin-Mean-Shift has only twice the overhead of isotonic regression (+0.1s vs +0.05s)? Since isotonic regression just needs to apply the mapping function from [0, 1] to [0, 1] to the inferred sample, while Bin-Mean-Shift needs to compute the distance from the new sample to all samples from the calibration set. The same question applies to mapping-based recalibration methods such as histogram binning (+0.03s). * Could you develop when to use the histogram-based version (section 5.2) and when to use the continuous version (section 5.1) of your method? --- Typos: * L148: it should be 80% instead of 85%. * L162: repeated word Appendix.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

The authors discuss the following limitations: * The proposed recalibration technique needs to maintain the calibration set during inference to compute the proximity of the new points. * Maintaining the calibration set for inference may challenge the method when memory is limited. * Focus limited to the closed-set multi-class classification problem.

Reviewer Wiui6/10 · confidence 3/52023-07-26

Summary

This work addresses the problem of proximity bias and confidence calibration by performing a comprehensive empirical study of various pretrained ImageNet models. The empirical findings provide insights on persistence of proximity bias even after performing calibration using existing post-hoc calibration algorithms. To mitiagte proximity bias and improve confidence calibration based on sample proximity, the paper proposes PROCAL algorithm that can be used as a plug-and-play method combined with existing calibration approaches. Further, proximity-informed expected calibration error metric is introduced to quantify the effectiveness of calibration algorithms in mitigating proximity bias.

Strengths

- A comprehensive study of pretrained ImageNet models involving various neural network architectures on their model calibration and proximity bias evaluation. The empirical study is performed on image classification tasks (under balanced, long-tail, and distribution-shift settings) and text classification tasks. - Experimental evaluation if thorough using various calibration metrics. - The paper provides many interesting observations from the empirical study related to proximity bias and model calibration, which is an important area of study under long-tailed data distribution settings.

Weaknesses

- The presentation of experimental results and writeup of Experiments Section 6 can be improvised, the findings for many of the questions are pointed to the Appendix without any brief details in the manuscript. I understand this is due to page limitation, but I would suggest the authors to focus on the results that are presented in main manuscript, or include the brief info in the manuscript at least.

Questions

- Are the iNaturalist 2021 results in Table 1 based on transfer learning from pretrained ImageNet models, or directly evaluated on pretrained backbone? - What are the criteria to choose between density-ratio calibration versus Bin-Mean-shift approach? - typo in line#309 ImageNet-TL --> ImageNet-LT

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors have addressed the limitations of their work in the manuscript.

Reviewer 4HP92023-08-11

Thank you for your response

I thank the reviewers for the responses to my and other reviewers' questions and suggestions. It seems that on the whole the final paper will be quite similar to the version we reviewed with some additional clarifications. I do think the question of domain shift could be better explored (or explained). For a concrete real-world example, to use the skin color/cancer detection example. If the samples in the training data are collected from majority light-skinned individuals, the ProCAL method would likely help adjust the confidences for the dark-skinned minority. However, if the training data are collected from exclusively light-skinned individuals, can the ProCAL method still be effective? Overall, I keep my score, and thank the authors for an interesting paper.

Authorsrebuttal2023-08-15

Experiments show that ProCal effectively improves upon existing algorithms in the domain shift setting

Thank you for the prompt reply and the insightful question! To explore the issue of domain shift, where test data has a shifted distribution not seen in training, we conduct experiments using **ImageNet (training set)** and **ImageNet-Sketch (test set)**. The datasets are chosen because all ImageNet images are real-world photos, while all ImageNet-Sketch images are sketches, collected using Google Search with class label keywords and "sketch", similar to the case of skin color example. **Experiment Setup**: We employ a ResNet50 backbone from `timm`, trained on the ImageNet. Then we train our ProCal using the validation set from `ImageNet` and tested it on the 50,000 images from `ImageNet-Sketch`. The term `base` represents the corresponding method's outputs (e.g. Conf, TS, ETS), while "+ours" indicates the output confidence scores corrected using our approach. | Method | ECE | ACE | MCE | PIECE | | :--- | :--- | :--- | :--- | :--- | | | base / +ours | base / +ours | base / +ours | base / +ours | | Conf| 0.0871 / **0.0176** | 0.0869 / **0.0174** | 0.0365 / **0.0062** | 0.0885 / **0.0255** | | TS | 0.0501 / **0.0180** | 0.0501 / **0.0170** | 0.0153 / **0.0025** | 0.0524 / **0.0223** | | ETS | 0.0457 / **0.0192** | 0.0479 / **0.0192** | 0.0067 / **0.0024** | 0.0494 / **0.0221** | | PTS | **0.0126** / 0.0129 | **0.0125** / 0.0132 | **0.0019** / 0.0023 | 0.0208 / **0.0188** | | PTSK| **0.0094** / 0.0114 | **0.0096** / 0.0126 | **0.0011** / 0.0019 | 0.0275 / **0.0191** | | MIR | 0.0194 / **0.0154** | 0.0193 / **0.0155** | 0.0032 / **0.0030** | 0.0243 / **0.0240** | **Results**: As demonstrated in the table above, ProCal effectively improves upon existing algorithm in many cases. While we observe a slight increase in ECE/ACE/MCE when ProCal is paired with with PTS and PTSK, this is probably attributed to the original methods suffering from the "cancellation effect”, where positive and negative calibration errors within the same confidence bin cancel out each other (see responses to Q5 above; or lines 170-181). Under the PIECE metric that captures the cancellation effect, our method consistently outperforms all methods by large margins, and effectively mitigates their proximity bias.

Reviewer 1D752023-08-17

I thank the authors for their detailed rebuttal. Concerning the answer to Q1 on the choice of the $\lambda$ coefficient, according to the authors, it appears that $\lambda = 1$ can be problematic when there are too few samples in the bins. I wonder if this work could benefit from the derivations of [31] section C.5. They face the same problem when estimating the inter-region variance when too few samples are in the regions (see the $GL_{plugin}$ curve in Fig. 1.a. which is overestimated when the number of samples per region is low, versus the $GL_{LB}$ curve, i.e., the $GL_{plugin}$ curve corrected with debiasing, which is correctly estimated even when the number of samples per region is low). I would guess that a debiasing similar to the one of section C.5 [31] could be applied to this work, which could enable values of $\lambda$ closer or equal to 1, even when a few samples are in the bins. Answer to Q2: the computational efficiency of the neighborhood search is quite impressive. Answer to Q3: Thanks for clarifying and revising the paper. [31] Alexandre Perez-Lebel, Marine Le Morvan, and Gaël Varoquaux. Beyond calibration: estimating the grouping loss of modern neural networks. In ICLR, 2023.

Reviewer 4gc82023-08-17

These clarifications are useful, thank you (it would also be useful to include some of these in the main body of the revision) - I keep my score.

Reviewer Wiui2023-08-17

I thank the authors for the rebuttal. I had read the responses and I retain my score.

Program Chairsdecision2023-09-21

Decision

Accept (spotlight)

© 2026 NYSGPT2525 LLC