Calibrating Neural Simulation-Based Inference with Differentiable Coverage Probability

Bayesian inference allows expressing the uncertainty of posterior belief under a probabilistic model given prior information and the likelihood of the evidence. Predominantly, the likelihood function is only implicitly established by a simulator posing the need for simulation-based inference (SBI). However, the existing algorithms can yield overconfident posteriors (Hermans *et al.*, 2022) defeating the whole purpose of credibility if the uncertainty quantification is inaccurate. We propose to include a calibration term directly into the training objective of the neural model in selected amortized SBI techniques. By introducing a relaxation of the classical formulation of calibration error we enable end-to-end backpropagation. The proposed method is not tied to any particular neural model and brings moderate computational overhead compared to the profits it introduces. It is directly applicable to existing computational pipelines allowing reliable black-box posterior inference. We empirically show on six benchmark problems that the proposed method achieves competitive or better results in terms of coverage and expected posterior density than the previously existing approaches.

Paper

References (50)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer JSnb6/10 · confidence 4/52023-07-02

Summary

The paper presents a method to perform calibrated simulation-based inference. To do so, the paper employs the well-known coverage and proposes a way to differentiate through this term and to use it as a regularizer during training. The authors evaluate their method on benchmark tasks and conclude that it has good coverage and (sometimes) even outperforms existing methods in terms of log-likelihood.

Strengths

**Originality**: The method is novel and the use of a differentiable sorting algorithm for differentiating through coverage is novel and useful. **Quality**: The theoretical part of the paper is done rigorously and the paper provides additional empirical results for the impact of hyperparameters and computational cost. **Clarity**: The figures are clear and support the messages of the paper.

Weaknesses

**Quality**: I expect that the method is very expensive if the batch size is large because in this case, GPU will not help either. This should be clarified in the paper. I found it very interesting that CA1NPE outperforms NPE in terms of log-likelihood. What exactly are the methods that the authors use to prevent overfitting? Always training for 500 epochs is clearly not something anybody would do in practice (L497). Please use a proper implementation of NRE or NPE to draw comparisons. The poor results of Appendix Fig 6 should be mentioned in the main paper and it should be highlighted that the method can be used to produce conservative posteriors, but that it is not suitable to produce calibrated posteriors. All tasks in the paper are very low dimensional. I would expect the NPE version of this algorithm to scale to high-dimensional asks, but this would need emprical evidence. For NRE, I could imagine that Importance sampling requires exceedingly many samples in high-d parameter spaces. Please clarify and ideally add tasks with a more high-dimensional parameter space. **Clarity**: Twice (L159 and L183), the authors propose alternative formulations of their method. They never empirically investigate these formulations and also do not describe why they are less good. I would appreciate if the authors either add additional details on these methods or remove them entirely to avoid confusion. The paper introduces **many** symbols which makes the paper very tedious to read. I would appreciate if the authors would redefine symbols in new sections to make the paper easier to follow. Also, some abbreviations do not really have to be defined (rarely used, e.g. KS, SAE, STE). Many papers are listed as `arxiv` although they got published. Please fix.

Questions

Why is the sorting based-computation preferrable over the direct computation? The authors say that `we need to backpropagate through F_N(\alpha_k)`. Is this a problem in practice? L472: what do you mean by “proposed regularizer set to calibration objective” L505: what does “outstanding values in observations” mean?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

2 fair

Contribution

3 good

Limitations

The authors state limitations of their method, but some limitations have to be highlighted (see comments above, compute time for large batch sizes, poor performance for calibrated (not convservative) posteriors.

Reviewer KAKy7/10 · confidence 3/52023-07-03

Summary

The paper proposes a calibration term to be used directly in the training objective of NREs and NPEs. The paper shows that the introduction of this term achieves competitive or better results in terms of coverage and expected posterior density.

Strengths

* The quality of the writing and presentation is high, with the structure easy to follow. In particular the related work is clearly included in the introduction. * The topic is of importance as it is increasingly more common to use neural estimators in SBI applications. * The experimentation is sufficient to show the effect of incorporating the new regulariser. It seems to show that the new regulariser leads to slightly conservative estimators that are on average better calibrated than the baselines.

Weaknesses

* The paper does not appear to have any major weakness. A few minor weaknesses that seem to exist have been appropriately described in the paper. These are the computational cost of the approach and the fact that the paper’s main metric of performance is also the same one that has been used in the regulariser. If the authors could think of a different metric to use to evaluate performance then the paper would be further strengthened. However, it is appreciated that finding a new metric would potentially be a new paper in its own right. Perhaps including C2ST as a metric in the appendix might provide additional comparison. While the focus of the paper is on calibration, it is possible to get a perfectly calibrated, but badly performing estimator. * There is no mention of hyperparameter optimisation (although a sensitivity analysis is given).

Questions

* Line 206: Does IS provide an improvement for NPE, even when it is not needed? * Since the regulariser relies on importance sampling did the authors do any experimentation on how the approach scales with dimension of $\theta$? This was mentioned in future work, but even in the experiments in the paper the theta varies across experiments. What are the dimensions of the experiments included in this paper? And did the authors see that the higher dimensional ones performed worse with the new regulariser?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

* The limitation related to computational cost is sufficiently highlighted in the work.

Reviewer Lpwd7/10 · confidence 4/52023-07-06

Summary

The authors suggest a new objective function for simulation-based inference that adds a penalty term to the “expected score” objective that is used by many other works. This penalty term encourages the resulting posterior approximations to be well-calibrated in the sense that the $1-\alpha$ highest probability density regions contain the ground-truth parameter $100(1-\alpha)$% of the time. The penalty term is an extension of the Kolmogorov-Smirnov test statistic for goodness of fit between a $U(0,1)$ distribution and the rank statistics of eq. 7, which are asymptotically uniformly distributed when $p(\theta \mid x)$ is calibrated. Minimizing this penalty term alongside the usual objective function should yield posteriors that are calibrated while substantially different from the degenerate case of the prior (which is trivially calibrated).

Strengths

* The proposed penalty/regularization term is relatively lightweight computationally, and can be tacked on to many existing simulation-based inference algorithms. * The intuition is simple; the construction of the statistics $\hat{\alpha}$ is clever, and the Kolmogorov-Smirnov test statistic is well-understood. * The method is assessed on a variety of problems from the test suite of benchmarks provided by Lueckmann et al. (2021), and performs favorably in that the resulting posteriors are either calibrated or tend to be conservative.

Weaknesses

* Although synthetic likelihood and ABC approaches are mentioned in the introduction, the proposed method seems prohibitively costly in these scenarios, and seems geared toward amortized methods only, where eq. 7 can be computed rapidly. * For ratio estimation in particular, the non-regularized NRE is better calibrated than the corrected version in some cases (e.g., spatial SIR), suggesting that it is at least possible the penalty term can worsen calibration. * In practice, it appears that attempts at calibration often result in conservative rather than calibrated posteriors; while this is likely still preferable to many users to the alternative of *not* adding this correction, it suggests a bit of a misnomer.

Questions

* The method called neural posterior estimation (NPE) has been shown in other work to be equivalent to minimizing a forward KL divergence between exact and approximate posteriors (Reweighted Wake-Sleep (Bornschein and Bengio, 2015), and a similar work “Revisiting Reweighted Wake Sleep…” (Le et al., 2018)); it’s been argued in these that the resulting neural posterior estimates tend to be overdispersed or conservative as a result. 1) How does this impact the relevance of this work if the baseline is usually already conservative? 2) The NPE row of Figure 1 doesn’t seem to reflect this intuition in practice; is there any explanation for why this is so? Is NPE indeed the same as sleep-phase in reweighted wake-sleep in the experiments? * What are the advantages and disadvantages of using either the sorting-based computation or the direct computation method? Is the use of the latter solely due to the belief that backprop through indicator functions is somehow worse? As $\hat{\alpha}$ itself requires indicator functions, I suppose the backprop operation can’t be too problematic. It would be interesting to see both methods implemented in the experiments. * The form of eq. 9 suggests self-normalized importance sampling, but as $\hat{p}(\theta \mid x^*)$ is already normalized, this doesn't seem necessary. Maybe this is a misunderstanding of notation. To my reading, though, just the numerator of eq. 9 is a valid importance sampling estimator. Is so-called “standard” or self-normalized importance sampling nonetheless used?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

Generally the settings where this work may be applied are clear, but as mentioned above discussing the use of the method in amortized vs. non-amortized settings may make more clear the costs associated with with this method when $\hat{p}(\theta_j \mid x^*)$ can’t be computed quickly.

Reviewer T8qr7/10 · confidence 3/52023-07-27

Summary

The paper introduces a training algorithm for posterior distribution learning in the likelihood free setting that combats overconfident models. The authors focus on the expected coverage probability (ECP) from Hermans et al., 2022 to measure if a model posterior is conservative. Specifically, when its value is equal to the credibility level used in its calculation, then the posterior is calibrated. An equivalent condition is that the distribution of credibility levels constructed from samples (lemma 1) is uniformly distributed. With this in mind, the authors propose a regularizer that penalizes how far the distribution of these credibility levels are from a uniform distribution. The main contribution of the paper is the algorithm used to train with this regularizer. The authors use a variety of techniques in their algorithm including a one-sample Kolmogorov-Smirnov test to work with finite samples, differentiable sorting to implement the test, and importance sampling to choose useful samples to use. The experimental results demonstrate that the learned models are more conservative than their unregularized counterparts while not sacrificing performance in terms of likelihood. Furthermore, the authors showed the effect of the hyperparameters on performance and also showed the that the training algorithm can be expensive if not run on GPUs.

Strengths

- Directly addresses the "trust crisis in simulation-based inference" described in Hermans et al., 2022. - Strong empirical results to demonstrate that the training algorithm works. - The paper progressed smoothly from the background to the proposed algorithm. - The layout of the text and plots are visually easy to digest.

Weaknesses

- It wasn't too clear to me after reading the main text why having uniformly distributed credibility levels was the right thing to want. - Similarly, it took a bit of drawing to see how all of the values in section 2 were related to each other. Adding something like figure 9 to the main text could greatly help introduce the background material.

Questions

- Why choose the one-sample Kolmogorov-Smirnov test to measure the divergence over others?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Yes, the authors acknowledge that their method is computationally expensive and may not scale to high dimensions as is.

Reviewer JSnb2023-08-13

Response to rebuttal

Thank you very much for the detailed response. It clarifies most of my concerns, and I have increased my score to 6. In my opinion, the main limitation remains that the method is only being demonstrated on very low-dimensional parameter spaces, in particular because importance sampling could lead to issues in more high-D parameter spaces. It would be amazing if the authors could add such analyses for the camera ready version, but I do think that this is a strong paper either way. Congrats on this nice work!

Reviewer KAKy2023-08-14

Response

Thanks - confirming that I have read the rebuttal.

Reviewer Lpwd2023-08-16

Rebuttal response

Thanks to the authors for the detailed point-by-point response. My main question about the instances of over-confidence (i.e. lack of conservativeness) for NPE has been answered by the use of a fixed dataset for training. As this has been reproduced from and discussed extensively in Hermans et al., I’m satisfied with these. The authors have indicated that they will make more clear the distinction between amortized and non-amortized approaches as well, which is appreciated. My primary remaining critique is on the use of “calibration” as the selling point of the paper. While I agree that conservative posteriors are beneficial, they can also be obtained trivially (e.g., take the prior), and I’m not sure I agree with the authors that “... we make no claim that the proposed method is able to systematically provide [calibrated posteriors]” due to the title. The authors have indicated that they will address this in the exposition and I hope this is done. A new concern I have noticed is that in the experiments, a higher computational budget seems to result in more conservative posteriors sometimes. In particular, I’m looking at the M/G/1 column of Figure 1 in the main body. In the “Cal” versions, the yellow line corresponding to the highest computational budget is either the most or second-most conservative on average. I would appreciate the authors’ comments on this point. This compares unfavorably with, say, BNRE which seems to maintain conservativeness, while becoming more and more calibrated as the computational budget increases (e.g., in the SIR plot for BNRE).

Authorsrebuttal2023-08-18

- "...higher computational budget seems to result in more conservative posteriors sometimes..." To address the question we will use Figure 5 located in the B section of the Appendix. There, one finds that BNRE indeed tends to yield AUC closer to zero as the simulation budget increases, while our proposed regularizer (CalNRE and CalNPE rows) does not exhibit such a behavior. We consider it to be consistent with the properties of the proposed solution when the conservativeness error is minimized (results presented in Fig 1, 2, and 5), because once the model yields conservative posteriors there is no penalty from the regularizer, and thus the training objective does not distinguish between "more conservative" and "less conservative" models. It turns out, that in such a regime the main loss term favors less-confident models when exposed to more training data. This may seem counter-intuitive at first (maximizing predictive performance -> minimizing under-confidence) but is the desired phenomenon when the inverse problem is ambiguous (ground-truth posterior is not a Dirac delta), because a higher simulation budget means exposure to more diverse samples that should result in less-confident models. In fact, this is exactly what we observe for the non-regularized models, therefore we conclude this is a behavior of the NRE/NPE approach itself. If bringing the AUC as close to zero as possible is desired we hypothesize that adding a low-weighted calibration error term in the late phase of training (or adding it progressively) could bring some improvement. However, we did not conduct such experiments and now see it as a potential future work direction.

Reviewer T8qr2023-08-16

Thanks, that makes sense. I don't have anything other questions, the contributions are relevant and clearly presented in the paper.

© 2026 NYSGPT2525 LLC