Summary
In the paper, the authors propose a method to calibrate image classifier probabilities using perturbations in the input image or hidden features, instead of other popular calibration methods such as various versions of Temperature Scaling. On several image classification benchmarks (e.g., CIFAR and ImageNet), the authors demonstrate that the final recalibrated predicted probabilities—generated by sampling labels from predicted class distributions for multiple perturbed image versions and then averaging them to recompute new class probabilities—can achieve low calibration errors (e.g., ECE) and outperform some calibration baselines.
Strengths
* Topic model calibration is important within the domains of robustness, safety, and uncertainty, especially considering recent developments in deep learning.
* The paper is clearly written and easy to follow, with an intuitive and easily understandable idea. The related work section provides sufficient discussion of existing calibration metrics.
* The approach does not require architectural changes to the model.
Weaknesses
* **the paper overlooks a significant body of relevant literature.** For example, the idea developed in [1] to generate uncertainty by running several inferences with perturbed inputs is very similar to the approach proposed in this paper, as are other sensitivity-based methods such as [2], which shares the same motivation as the proposed method, significantly undermining the paper's novelty. Additionally, [3], mentioned in the appendix, bears a strong resemblance to the proposed method. Given this, positioning the method as a novel approach to calibration or uncertainty estimation seems overstated.
* **the experiments lack relevant baselines.** Although the authors propose a version of the method where noise is added to intermediate hidden representations, the original formulation—where noise is applied to the inputs—requires multiple inferences. In this context, ensembling-based approaches (some of which are mentioned in the appendix) should be considered as relevant baselines and should not be omitted from the experiments. Moreover, it is unclear how the inference-time vs. calibration-quality trade-off is addressed in the proposed method; some existing ensembling approaches, such as Deep Ensembles, MC-Dropout, and BatchEnsembles, may require additional computation but consistently demonstrate the highest uncertainty quality [4] (in terms of both calibration and epistemic uncertainty). Additionally, recently introduced efficient ensembling approaches, such as PackedEnsembles [5], Depth Uncertainty [6], or MixMo [7], require only a single inference to generate multiple predictions and estimate ensembling-based uncertainty, thereby mitigating computational overhead while maintaining the high uncertainty quality of ensembles.
* related to the previous point, **the method is specifically designed with image classification in mind**, making both its design and the conducted experiments quite limited. The original formulation, which involves adding perturbations to the input, is not feasible for other popular domains, such as language modeling, where adding Gaussian noise is not feasible. Additionally, certain architectures, such as GNNs, are also not suitable for this approach. Using hidden representations to overcome these limitations appears more like an ad hoc solution to issues of computational overhead and broader applicability rather than a well-designed solution. A potential drawback of this version is the potential degradation of model performance (e.g., accuracy), as it resembles Dropout (or Gaussian Dropout [8]), which is known to degrade model performance [4].
* the following example illustrates the **potential ineffectiveness of the proposed approach**: consider the setup in Figure 1(b) and a point, such as (2, -4), which lies deep within the predicted class (class 0) but should have higher predicted uncertainty, as it is not near the center of the red class. Adding Gaussian noise to this point won’t change the predicted class, so the recalibrated predictions would remain highly miscalibrated. A similar issue could occur when the predicted softmax vector changes under different perturbations, but the argmax class remains nearly the same, resulting in a degenerate distribution and miscalibrated predictions. By contrast, ensembling approaches rarely exhibit this behavior.
[1] Mi, Lu, et al. "Training-free uncertainty estimation for dense regression: Sensitivity as a surrogate." AAAI 2022.
[2] Durasov, Nikita, et al. "Zigzag: Universal sampling-free uncertainty estimation through two-step inference." TMLR 2024.
[3] Conde, Pedro, et al. "Approaching test time augmentation in the context of uncertainty calibration for deep neural networks." arXiv 2023.
[4] Ashukha, Arsenii, et al. "Pitfalls of in-domain uncertainty estimation and ensembling in deep learning." ICLR 2020.
[5] Laurent, Olivier, et al. "Packed-ensembles for efficient uncertainty estimation." ICLR 2023.
[6] Antorán, Javier, et al. "Depth uncertainty in neural networks." NeurIPS 2020.
[7] Ramé, Alexandre, et al. "Mixmo: Mixing multiple inputs for multiple outputs via deep subnetworks." ICCV 2021.
[8] Gal, Yarin, and Zoubin Ghahramani. "Dropout as a bayesian approximation: Representing model uncertainty in deep learning." ICML 2016.
Questions
As it was mentioned in the Weaknesses section, the paper:
1) misses a significant body of important related work and lacks novelty (which I find to be a core issue), as sensitivity-based approaches have been around for a long time.
2) lacks thorough experimental validation to support the idea that the method is indeed extendable and effective.
3) the approach has important theoretical issues that were not addressed in the method discussion.
To name a few questions:
1. How does the proposed method distinguish itself from prior perturbation-based uncertainty methods? Could its unique contributions be clarified?
2. Why were ensembling methods not included as baselines? Would a comparison, especially with efficient ensembling variants, provide a fuller performance picture?
3. How generalizable is the method beyond image classification? Could adaptations for language models or GNNs improve its applicability?
4. Have the theoretical limitations of the method been considered, and could further analysis address potential weaknesses in the approach’s foundations?