Optimizing proper loss functions is popularly believed to yield predictors with good calibration properties; the intuition being that for such losses, the global optimum is to predict the ground-truth probabilities, which is indeed calibrated. However, typical machine learning models are trained to approximately minimize loss over restricted families of predictors, that are unlikely to contain the ground truth. Under what circumstances does optimizing proper loss over a restricted family yield calibrated models? What precise calibration guarantees does it give? In this work, we provide a rigorous answer to these questions. We replace the global optimality with a local optimality condition stipulating that the (proper) loss of the predictor cannot be reduced much by post-processing its predictions with a certain family of Lipschitz functions. We show that any predictor with this local optimality satisfies smooth calibration as defined in Kakade-Foster (2008), Błasiok et al. (2023). Local optimality is plausibly satisfied by well-trained DNNs, which suggests an explanation for why they are calibrated from proper loss minimization alone. Finally, we show that the connection between local optimality and calibration error goes both ways: nearly calibrated predictors are also nearly locally optimal.
Paper
Similar papers
Peer review
Summary
This paper elucidates the relationship between proper losses and calibration by providing the minimal necessary and sufficient condition for proper losses to induce calibrated models. The condition, local optimality, delineates the concept that no post-processing functions can improve a proper loss anymore. This is related to a specific measure of calibration called smooth calibration error, which is a kind of correlation between miscalibration and model predictions. The smooth calibration error is used here because it does not suffer from a discontinuous nature, unlike the popular expected calibration error, and it naturally emerges from the Bregman divergence structure. By leveraging the Bregman divergence structure of proper losses and convex analysis, a theoretical connection between the smooth calibration error and the post-processing gap (the quantitative version of the local optimality condition) is established, which indicates that minimizing the post-processing gap is necessary and sufficient to achieve sufficiently small calibration error. As an implication of the theory, the authors point out a potential connection between the implicit bias regularization of DNNs and the post-processing gap so that sufficiently over-parametrized networks may achieve a small post-processing gap, leading to the current well-calibrated neural networks. Overall, this paper pushes the understanding of proper losses and calibration toward the context of modern neural network regimes.
Strengths
- A new connection between proper losses and calibration: Though the two concepts seek similar goals, they have been studied independently, and the relationship has not been understood well. The main theorem of this paper draws the connection by establishing the upper and lower bounds of the post-processing gap (related to proper losses) by the smooth calibration error (related to calibration). As far as I know, this is the first attempt to connect the two concepts. The concept of the post-processing gap is well motivated by deep neural networks (regarding the post-process as the layer addition). - A transparent proof: The proof of the main result (Theorem E.8) gives us a nice picture of the relationship between the calibration error and proper losses. Specifically, the proof of the bounds mostly leverages the smoothness of a function $\\psi$. This proof is simple and gives us an insight that the structure of $\\psi$ essentially governs the connection. - The clarity of the presentation: Though the concepts introduced in this paper are rather dense, the authors did a nice job of presenting them gradually from a conceptual level to a technical level, which helps readers who may not be familiar with those concepts understand them easily.
Weaknesses
- Potential gaps between the theory and Guo et al. (2017): The authors argue that "the previous generation of image classifiers were poorly calibrated [Guo et al., 2017]" and "state-of-the-art DNNs are often well-calibrated: because their test loss cannot be improved much by adding a few more layers." However, I feel that the architectures used by Guo et al. (2017) are already very deep. For example, in their pilot study (Figure 1), they chose to use a 110-layer ResNet, which would be sufficiently over-parametrized. Since Guo et al. (2017) argued that DNNs are poorly calibrated even with that number of layers, I would like to see the authors' discussion on this line. - Missing key references: Some results and claims in the paper are substantially related to previous works that are not mentioned in the paper. It would be great to give credit to them. For example, the dual mapping of the form $\\mathsf{dual}(v) := \\ell(0,v) - \\ell(1,v)$ (l. 299) is known in Eq. (47) of Buja et al. (2005); The composition of a proper loss $\\ell$ and the dual mapping $\\psi$ is known as composite losses, coined in Reid and Williamson (2010). The dual loss form in Eq. (6) and Definition 4.3 is known as the Fenchel-Young losses, and the expression was shown in Eq. (14) of Blondel et al. (2020) and Eq. (11) of Duchi et al. (2018); The convex conjugate structure $\\nabla\\psi(\\mathsf{dual}(v)) = v$ (l. 329) was pointed out in Figure 1 of Bao and Sugiyama (2021); Some parts of Lemma E.4 are closely related to Proposition 2 of Blondel et al. (2020). - Restriction to binary classification: The attention of this whole paper is restricted to binary classification, as mentioned in the conclusion. This is far more restrictive in practical situations. But I don't think this limitation undermines the impact of this paper. **References** - Reid and Williamson (2010) "Composite Binary Losses" - Buja et al. (2005) "Loss Functions for Binary Class Probability Estimation and Classification: Structure and Applications" - Blondel et al. (2020) "Learning with Fenchel-Young Losses" - Duchi et al. (2018) "Multiclass Classification, Information, Divergence, and Surrogate Risk" - Bao and Sugiyama (2021) "Fenchel-Young Losses with Skewed Entropies for Class-posterior Probability Estimation'
Questions
Some minor comments: - (Question) At l.261, I don't get the point of "within the restricted class where the logit is a linear combination of the features," specifically, what "the features" mean in the current context. Could you clarify? - (Question) Why do you present the non-generalized dual calibration error in Section 4, unlike the generalized one in the appendix? I do not get the reason why it is useful to confine the generalization $w(x,g(x))$ to $\\eta(x)$. - (Typo) At l.75, "depends on on the architecture" -> "depends on the architecture" - (Typo) At l.204, a parenthesis is missing for $\\eta(f(x))$.
Rating
9: Very Strong Accept: Technically flawless paper with groundbreaking impact on at least one area of AI/ML and excellent impact on multiple areas of AI/ML, with flawless evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
4 excellent
Presentation
4 excellent
Contribution
4 excellent
Limitations
The authors discuss the limitations of this work at the end of the paper and point out that there is room to investigate the connection among calibration, DNN architectures, and optimization. Most of the work is theoretical, and few negative societal concerns would apply.
Summary
This work introduces a local optimality condition for models (with respect to proper losses) based on (additive) post-processing with a 1-Lipschitz function that is necessary and sufficient for calibration. The authors also connect their results to the idea of implicit regularization, showing that structural risk minimization with an appropriate class of complexity measures achieves good calibration.
Strengths
## Originality The introduced notion of post-processing error and its connection to (smooth) calibration is novel and useful, as it proves plausible explanations for why current SOTA deep learning models are better calibrated when compared to previous generations. Additionally, the introduced local optimality condition based on post-processing is distinct from conditions seen in optimization, but remains non-trivial since the class of post-processing functions is restricted to be 1-Lipschitz. ## Quality The paper is technically sound and polished; the definitions are well-motivated and simpler results are accompanied with proofs (or at least intuitive justification) in the main body of the paper. ## Clarity The theory is easy to follow, and the authors have done a good job of scaling the complexity of the results as the paper progresses (introducing simple examples and high-level ideas first, and generalizing later). The authors also qualitatively connect their results to recent empirical phenomena in deep learning, which is helpful for grounding the theory. ## Significance Calibration is an increasingly important problem, and the paper provides insights into how to think about the relationship between calibration and post-processing procedures.
Weaknesses
- *Experiments:* While I understand this is a theory paper, I do think it would be nice to have some experimental analysis of whether the local optimality condition is satisfied for modern architectures (i.e. simply adding an extra layer as suggested in the paper and analyzing calibration performance). - *Addressing Popular Post-Processing Methods:* If I understand the discussion around Definition 2.2 correctly, the post-processing operation defined does not include temperature-scaling-type techniques, since $f(x)$ includes the sigmoid operation and one would have to apply the re-scaling to the logits. If this is not the case, some clarification in this section linking the post-processing definition back to the standard post-processing approaches would be useful.
Questions
My only question is related to my discussion under weaknesses; namely how temperature scaling type methods would fit in with Definition 2.2.
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
4 excellent
Presentation
3 good
Contribution
3 good
Limitations
The authors appropriately address limitations; namely, they acknowledge that their results do not explain _why_ the optimization process for current models leads to calibrated predictors.
Summary
The paper considers calibration in binary classification when training was performed with proper losses. The authors showed that the post-processing gap of a predictor, which is a maximum improvement of the loss given any 1-Lipschitz update (calibration) function, could be both lower and upper bounded by a simple expression of smooth calibration error. Moreover, the authors proved that the constants used in upper and lower bounds are optimal.
Strengths
The paper describes the connection between proper loss landscapes and calibration error improvement, which is an important and quite original problem under the current formulation. The paper has good positioning among other papers in the field, with clear references to many studies and a detailed indication of what was their contribution. The clarity of the paper is good.
Weaknesses
The link with potential practical application is missing (or rather not convincing). Adding small-scale experiments to support theoretical results would be beneficial, e.g. to check whether the pre-requisites of the theoretical results are (approximately) fulfilled in practice. The discussion and explanation of theoretical results could be improved. The current impression is that either theoretical results are not that impressive (and based mostly on properties of Bregman divergence) or that the potential impact of derived theory is oversold.
Questions
1. The derived results are valid in binary classification settings. Will it be possible, and what are the possible limitations, to obtain similar results in multiclass settings? 2. Could it be said that all theoretical derivations are based on properties of Bregman divergences but in a new setting connected with calibration errors?
Rating
4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
The limitations have been covered at the end of the paper. No need to discuss societal impact in this paper.
Summary
The paper provides a novel characterization of calibration as local optimality of the predictor w.r.t the global loss under post-processing of the prediction through a class of functions. The paper proves that any predictor satisfying such a condition is smoothly calibrated in the sense of [Kakade and Foster, 2008, Błasiok et al., 2023] and vice-versa. The paper further provides arguments for why such a condition should be satisfied by Deep Neural Networks (DNNs).
Strengths
- The paper is well written, with a clear structure and ideas being developed in a coherent and logical sequence. - The topic is of large importance in the present context of research in machine learning and has numerous implications for practical applications. - The authors contribute a novel perspective on the 'implicit' calibration of machine learning models without the use of algorithms designed specifically for calibration, an aspect that is not extensively explored in current literature. - The explicit characterization of calibration in terms of local optimality under post-processing transformations, while implicit in earlier works, appears to be novel. This equivalence could be useful for further theoretical analysis of calibration, especially for deep neural networks. - The proposed framework is general and does not rely on specific choices of model's architecture or data-distribution. - The paper provides partial explanations for the observed calibration of deep neural networks trained on large training datasets.
Weaknesses
- **Limited technical contributions**: Claim 2.1 itself directly follows from the definition of perfect calibration. The main theoretical results in the paper are generalized formulations of Claim 2.1 to smooth calibration, lipschitz-post processing functions, and general proper losses. While these generalizations themselves are novel, their proofs involve straightforward algebra and convex analysis. Therefore, the technical and mathematical contributions of the paper are limited. The results can be strengthened from examples of non-trivial results that can be derived using the local-optimality based characterization of calibration. For deep neural networks, the present results are only suggestive and would benefit from additional details and concrete results. - **Missing references**: - On double-descent in uncertainty quantification in overparametrized models: Lucas Clarte, Bruno Loureiro, Florent Krzakala, Lenka Zdeborova, Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR 206:7089-7125, 2023. - Theoretical characterization of uncertainty in high-dimensional linear classification, Lucas Clarté, Bruno Loureiro, Florent Krzakala, and Lenka Zdeborová, Machine Learning: Science and Technology, Volume 4, Number 2 The above papers analyze calibration for empirical risk minimization and like the present paper, also highlight the role played by regularization. - **Dataset size and overparameterization**: The paper does not address aspects related to generalization and the effect of the training dataset size. While modern training setups utlize large training dataset sizes and one-pass SGD, their behavior in the proportional regimes of high-dimensional inputs and parameters is non-trivial and not equivalent to the minimization of population loss. For instance, the above papers establish a double-descent like phenomenon for calibration for varying levels of overparametrization. - **Experiments**: In light of the limited technical contributions, the paper could benefit from experimental justification of the validity of the local optimality condition for deep neural networks in realistic training setups.
Questions
- What are some complexity measures satisfying the condition in Claim 4.8? - What are the limitations of the definition of smooth calibration error used in the paper, especially in the context of deep neural networks?
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
4 excellent
Limitations
Yes, the limitations have been adequately addressed. The work is theoretical in nature and does not have direct societal impacts.
Summary
This paper seeks to explore and formalize the relationship between minimizing proper loss and calibration in machine learning models, particularly deep neural networks (DNNs). It presents a local optimality condition that is necessary and sufficient to ensure model calibration. The work discusses the implications of these findings on the calibration properties of modern DNNs and presents algorithms that can guarantee calibration. The paper also contrasts the differences in calibration between current and previous generation models through the lens of generalization.
Strengths
NA
Weaknesses
NA
Questions
NA
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
NA
"Sense" or "no sense"
Dear Reviewer, did you mean to say "That makes sense" or "That makes no sense"? In the case of the second, can you please specify what in particular? Thanks :).
Sorry for the typo, I meant "makes sense". My apologies for the delayed response.
Response
I thank the authors for addressing my concerns. Specifically, the clarification on the relation to Guo et al. (2017) was helpful. It seems to become less misleading to add such a discussion in the next version to clarify the difference of the practice of neural networks training during these few years.
I thank the authors for providing clear and comprehensive responses. I've raised my rating on the assumption that the authors will incorporate discussions about the above points and missing references into the revised version of the paper.
Re: rebuttal
Thanks for the clarifications! **Re: Re: practical applications.** You have written in your response that _"Practitioners (using sklearn for instance) are currently encouraged to believe that solely by minimizing proper loss within a restricted family of functions, they will get calibrated predictors (see the references in our paper)"_. I am not convinced that it is true. Your work refers to a paper published in 2011 about sklearn documentation. Back then, there might indeed have been such a belief about proper losses, given that no large NNs models existed. The up-to-date sklearn documentation about calibration does not seem to contain such encouragement. In my opinion, it is common knowledge that models tend to be uncalibrated, unless post-hoc calibration is used. **Re: Re: experiments.** Please comment on the two following points and questions. 1. Experimental evidence you are referring to is mainly about models trained with cross-entropy. However, your theoretical results are valid for all proper losses. Could we expect that new large NNs could be trained basically with any proper loss and have a nearly perfect calibration, given, for example, a slightly larger dataset/architecture, than existing models trained with cross-entropy have? 2. You claimed that your theoretical results provide an explanation of calibration differences between an old and new generation of NN models (due to a much larger dataset and more complex architecture). Those calibration differences are the main experimental evidence of your theory in the paper. However, it could probably be explained in a simpler way: proper losses should have well-calibrated results on a train set. Once train set is big enough (as for new generation of NN models), we could expect that train set empirical distribution is very close to ground truth population distribution, so the results on the test set (generated from this unknown population distribution) would be well-calibrated. Could you comment on this reasoning, and do you agree with it? Could you suggest other existing experimental evidence that supports your theoretical results as opposed to the above simpler explanation?
Thank you for replying! **Re: practical applications.** We believe our quoted statement is well-justified for the following reasons: First, our statement is not just about neural networks, it is about minimizing proper losses *within a restricted family of functions.* Logistic regression is an instance of such restricted minimization, and many sources claim logistic regression is well-calibrated primarily because it minimizes a proper loss. As mentioned in our paper, the sklearn documentation as of August 16 2023 says "LogisticRegression returns well calibrated predictions by default as it has a canonical link function for its loss, i.e. the logit-link for the Log loss" (from the url we cited: https://scikit-learn.org/stable/modules/calibration.html). **Our 2011 citation is for the Scikit-learn package itself, and we also cited the up-to-date documentation url.** The wikipedia page on Platt scaling (as of August 16 2023) also claims that logistic regression gives well-calibrated models: “(Platt scaling) has less of an effect with **well-calibrated models such as logistic regression**, …” (https://en.wikipedia.org/wiki/Platt_scaling). However, our analysis demonstrates that this is not true: logistic regression can be severely mis-calibrated, essentially because it is not minimizing over a sufficiently rich family of functions (see lines 42-50, and Appendix B of our paper). Logistic regression remains an extremely popular method among practitioners of machine learning and data science in the real world, even in this age of "large NNs", and thus we believe our statement on practical relevance is justified. Finally, regarding neural networks, we do not agree with the claim that "it is common knowledge that models tend to be uncalibrated, unless post-hoc calibration is used." Specifically, because it is not true: there are now many documented instances of modern neural networks being extremely well-calibrated out-of-the box, as we cited in our paper. These citations include, for example, the GPT-4 tech report (Figure 8, left, in: https://arxiv.org/abs/2303.08774). **Re: experiments.** 1. Yes, our theory does predict that minimizing the test loss for essentially any proper loss over a rich enough model family will yield a calibrated model. We observed this experimentally in an ongoing and upcoming work. 2. Regarding the potential simpler explanation: we agree with the logic of the explanation (see our response to Reviewer CHsP) and the explanation was proposed by Carrell et al., 2022 (cited in our paper). However, the premise that “proper losses should have well-calibrated results on a train set” is non-trivial to establish and it is a motivation of our work. This premise is not true in general: logistic regression can be poorly calibrated on its train set, despite minimizing a proper loss. It turns out that sufficiently large neural networks *are* in fact well-calibrated on their train sets, as observed experimentally by Carrell et al., 2022, and these observations were inspiration for our theoretical work.
First, the claim “(Platt scaling) has less of an effect with well-calibrated models such as logistic regression, …” from Wikipedia should not be interpreted as claiming some sort of a guarantee. It is obvious that (nearly) every model is miscalibrated since the requirement to be calibrated requires equality of real values which is almost surely not satisfied. The only question is how much data are needed to demonstrate miscalibration. Therefore, the claim of logistic regression being well-calibrated is an empirical one - in many practical applications people have observed that logistic regression tends to be relatively well-calibrated. But as you state in your Appendix B lines 536-537, "Similar examples of logistic regression being miscalibrated are found in Kull et al. [2017]." Note that your mentioned lines 42-50 give a false impression as if this 'discovery' were your novel contribution. Second, regarding your claim that "there are now many documented instances of modern neural networks being extremely well-calibrated out-of-the box" and your subsequent reference to the GPT-4 tech report. The report itself is much more modest by stating "Interestingly, the pre-trained model is highly calibrated (its predicted confidence in an answer generally matches the probability of being correct)." They don't seem to have run any statistical hypothesis test on this, so it might even be that the model is significantly miscalibrated. I agree that it is more calibrated than some other networks that we have seen in the past years, but it might still benefit from further posthoc calibration (unless checked and proved otherwise). For example, the two rightmost columns in Table 1 of the NeurIPS paper (Mukhoti et al 2020) https://papers.neurips.cc/paper/2020/file/aeb7b30ef1d024a76f21a1d40e30c302-Paper.pdf show that even though models trained with focal loss were shown in that paper to be surprisingly well-calibrated, temperature scaling with temperatures in the range from 0.9 to 1.1 still nearly always improved calibration further. This demonstrates that the model had still been miscalibrated, although to a much less extent than cross-entropy-trained models (keeping in mind that focal loss is not proper whereas cross-entropy is). Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip Torr, and Puneet Dokania. Calibrating deep neural networks using focal loss. Advances in Neural Information Processing Systems, 33:15288–15299, 2020. Therefore, I still stand by the view that it is common knowledge that models tend to be miscalibrated, unless post-hoc calibration is used. Miscalibration is here a continuum from mild to severe, but nearly always, post-hoc calibration has been shown to improve calibration, even if by just a small margin. Regarding the last point, I don't think you have responded to my question, "Could you suggest other existing experimental evidence that supports your theoretical results as opposed to the above simpler explanation?".
We disagree on many points here, but this discussion on OpenReview does not appear to be converging to a resolution. We will thus leave our paper and existing comments to speak for themselves. We thank the reviewer for their time.
Decision
Accept (spotlight)