Beyond Deep Ensembles: A Large-Scale Evaluation of Bayesian Deep Learning under Distribution Shift

Summary

This paper evaluates several BDL algorithm on real-world datasets (and real-world distribution shifts) in terms of generalisation (accuracy/correlation metrics) and uncertainty estimation quality (calibration/signed calibration).

Strengths

1. I think there is a need for benchmarking BDL techniques, and the paper tries to tackle that problem. 2. The point about batch normalisation leading to really poor o.o.d. data performance is interesting, and makes sense. This could be emphasised more as a contribution of the work (if it is not already known, I'm not sure!).

Weaknesses

Overall, I vote for rejection. I am not convinced that this paper is of interest to the community, and ready for publication yet. However, I am open to changing my mind if I see evidence from the authors. I've included suggestions and questions for the authors that would enable me to raise my score. It's possible I misunderstood key details; I am more than happy to be corrected. 1. I do not think the paper makes a substantial enough contribution to be accepted at NeurIPS. The contribution of the signed calibration metric is nice but small. If there was a clear takeaway recommendation from the experiment results, that would be a useful contribution to the community, but as I can understand, there isn't one. Despite this, the experiment section is mostly descriptive and does not provide greater insight and understanding about when one method is better than another. Either making the argument that the experiment section makes clearer and using the results to provide greater insight and understanding would help me increase my score. For example, the result that Bayes-by-Backprop performs very well here is interesting and surprising to me. Why is this the case? VI usually underfits, so does VI regularise to the base model more strongly? Which would be good in the pre-trained setup? This result contradicts other work ([1]) where VI often performs poorly and underfits. Please explain the contradiction. 2. Missing reference to Nado et. al [1]. The need for baselines is clear, but what does this work offer over this work? I'm quite surprised that I didn't see the reference in the paper. For what it's worth, I think the Nado et al. baselines are also limited (only every evaluating marignal uncertainty, nothing about sequential decision making), so I think good work remains to be in this domain, but I don't think the paper offers much over this work. Would you please explain to me, in clear and precise terms, the contribution of this submission relative to this work? 3. Missing baselines: give the point of this paper is the comparison (at least), it is important to be comprehensive in terms of the baselines covered. I would like to see SNGP, DUQ, and some frequentist approaches like temperature rescaling. This is a clear weakness of the paper. Further, not including function-space approaches is a major limitation: the paper claims to do a thorough investigation, and leaves out an important class of methods. 4. A key point of the paper is that existing evaluations e.g., on CIFAR-10-C are unrealistic because the distribution shift is constructed and not real world. I think this point is interesting, but I would like to see evidence to support it. For example, if one would make different (and importantly different) conclusions based on benchmarking on CIFAR-10-C vs on WILDS. Further, other BDL papers do look at WILDS datasets e.g. [4], in case you weren't aware of this. Evaluating on WILDS is not sufficient for novelty, in my view. 5. "modern single-model BDL algorithms approximate the parameter posterior better than deep ensembles ..." Well, for one, your evaluation looks at total variation in predictive space, so you can make no claim about the __parameter__ posterior. And, furthermore, recent work [5] shows that the HMC chains from the Izmailov et al paper have not converged well, so I don't think a claim about "approximating the posterior" is valid, you can only claim to approximate the HMC chains. And since the HMC chains perform badly out of distribution, it is not clear that lower total variation is better! See [5] on that, which argues that we might not want to be doing full network inference in the first place. 6. The writing of the experiment section is very poor. The figures aren't referenced in the text, the writing is mostly around the results, but offers little in terms of understanding, insight, conclusions, or discussion. I found this very hard to read, and I am left confused: what are your take away messages? What is the contribution of the work? 7. A claim is made "BDL is in many cases competitive with algorithms that are specifically designed for OOD generalisation" (Line 393,394). This does not seem to be supported: what benchmarks did you compare to that are specifically designed for OOD data? Minor points: - "typical deep neural networks are highly confident on o.o.d. data". I'd argue this is a bit different with large pretrained models. - Since your results used pre-trained model, [6, 7] are relevant citations because they justify pre-training in a Bayesian setting. [1] Nado, Zachary, et al. "Uncertainty Baselines: Benchmarks for uncertainty & robustness in deep learning." arXiv preprint arXiv:2106.04015 (2021). [2] Liu, Jeremiah, et al. "Simple and principled uncertainty estimation with deterministic deep learning via distance awareness." Advances in Neural Information Processing Systems 33 (2020): 7498-7512. [3] Van Amersfoort, Joost, et al. "Uncertainty estimation using a single deep deterministic neural network." International conference on machine learning. PMLR, 2020. [4] Daxberger, Erik, et al. "Laplace redux-effortless bayesian deep learning." Advances in Neural Information Processing Systems 34 (2021): 20089-20103. [5] Sharma, Mrinank, et al. "Do Bayesian Neural Networks Need To Be Fully Stochastic?." International Conference on Artificial Intelligence and Statistics. PMLR, 2023. [6] Shwartz-Ziv, Ravid, et al. "Pre-train your loss: Easy bayesian transfer learning with informative priors." Advances in Neural Information Processing Systems 35 (2022): 27706-27715. [7] Sharma, Mrinank, et al. "Incorporating Unlabelled Data into Bayesian Neural Networks." arXiv preprint arXiv:2304.01762 (2023).

Questions

See also the weaknesses for the list of questions. Some additional questions are: 1. What are the differences between WILDS and e.g., CIFAR-10, other than the "realistic" dataset shift? Would you be more satisfied with BDL people evaluating on Imagenet-C? 2. How are hyperparameters selected? 3. How do you tune the prior precision for the Laplace approximation? In my experience, it can be quite difficult to tune well. I tried to look in the appendix, but it said that I didn't have permission to view the file. 4. Line 328 says Bayes-by-Backprop has a "last-layer nature". Why? People perform BBB on the full network.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

1 poor

Contribution

1 poor

Limitations

Seems fine to me; see weaknesses.

Summary

This paper presents an empirical comparison between various Bayesian methods/approaches on out-of-distribution (OOD) data. The authors focus on non-MCMC based methods (such as Bayes By Backprop, SWAG, and Laplace approximation). Methods are compared on challenging OOD benchmarks from the WILDS collection and using several network architectures. All compared methods are evaluated in two flavours, first as a single-mode posterior and second in an ensemble of models. Results are somewhat inconclusive, different methods tend to work better in terms of calibration and generalization on different datasets.

Strengths

* I think that the paper makes a nice contribution to the community. It may be hard to pick a model that perform well on OOD data in the Bayesian model zoo, and this paper makes another step towards a better understanding of this question. * These types of projects are hard to manage. There are many choices to be made in terms of methods, datasets, evaluation metrics, etc. I think that the authors made good choices in all of these aspects. For instance, they picked methods that reflect typical Bayesian methods people use, and they focused on 2-3 evaluation metrics that capture both uncertainty quantification and generalization capabilities adequately. * For the most part, the paper is written well. The authors justify many of their choices. * Evaluating the ensemble version of single-mode-posteriors is a good idea and was mandatory in my opinion. * The results seem reproducible, fully experimental details were given and the code was provided.

Weaknesses

* I find it hard to understand the key takeaways. I think the authors should present the main conclusion from each experiment and the key takeaways from all experiments (e.g., inside a text box with some background color, or in bold). * As the results are inconclusive, e.g., Rank-1 VI is best calibrated on PovertyMAP, but less so on IWILDCAM, FMOW, and RXRX1, I believe it would have been beneficial to suggest a possible explanation (or perform an empirical analysis) for the reasons that make a method work better on some datasets and worse on others. * An important evaluation metric that is missing in my opinion is in terms of computational complexity (memory and time of each method). It may be important information when comparing the methods as well. * The authors evaluate last-layer Bayesian methods, which is important and great. An important method that I find missing is deep kernel learning [1, 2]. It is also a popular last-layer Bayesian method and, from my experience, it tends to work better than most of the compared methods. * In terms of exposition: * I think that the authors should explain at the beginning of Section 5 or 5.1 that the analyses of the results are given in Sections 5.3 and 5.4. It is not clear until reaching there. * In the figures, the font of the tick labels, axis labels, and method names should be larger. [1] Wilson, A. G., Hu, Z., Salakhutdinov, R., & Xing, E. P. (2016, May). Deep kernel learning. In Artificial intelligence and statistics (pp. 370-378). PMLR. [2] Calandra, R., Peters, J., Rasmussen, C. E., & Deisenroth, M. P. (2016, July). Manifold Gaussian processes for regression. In 2016 International joint conference on neural networks (IJCNN) (pp. 3338-3345). IEEE.

Questions

* I may be wrong, but from the main text it seems that you evaluated BBB on the layer only, why not use it on the full network? * The reference format is a bit odd. For instance, not all authors names are written. Please check that you adhere to NeurIPS guidelines.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

The authors addressed limitations of their work.

Summary

This paper presents a systematic comprehensive evaluation of a wide range of scalable Bayesian deep learning (BDL) algorithms in distribution-shited real-world data scenarios. A signed version of calibration error is presented, which allows for the identification of overconfidence and underconfidence rather than only absolute calibration. The study bolsters the ongoing effort in the BDL community to establish the practicality of BDL algorithms in safety-critical settings.

Strengths

In my opinion, this line of work has huge potential but is in dire need of a "reality check" in order to make tangible, impactful progress in real-world safety-critical settings. I believe this paper makes real progress in that direction. Indeed, the primary motivating statement behind uncertainty estimation is one which appeals to its utility in safety-critical applications, i.e. how and when can we trust our predictions. However, perhaps ironically, much emphasis has been placed on reiterating the former aspect by proposing more sophisticated uncertainty estimation algorithms, and not enough emphasis has been placed on establishing their utility and reproducibility in the very safety-critical settings they were designed for. The paper is well-written and I found the empirics to be thorough and convincing. I believe this work will help guide the BDL community toward more practicable outcomes so I recommend acceptance.

Weaknesses

One weakness of this type of empirical review is that - although appropriately broad and comprehensive for a conference submission - the empirical study can always be extended to other tasks which may or may not change the conclusions. Another weakness is the lack of novelty from a methodological perspective; although I do appreciate the proposed usage of a signed calibration error to identify under- or overconfidence without having to look at reliability diagrams. I would suggest including an itemized list of the main takeaways from the empirical study in the conclusion/discussion section of this paper. I think this would help organize the findings of the study and would provide easy-to-follow actionable advice for BDL practitioners going forward.

Questions

-Can you summarise the implications for practitioners who use large-scale models and need to make decisions under distribution shifts?\ -Do you believe there is an inherent trade-off between accuracy, efficiency and calibration? \ -In your opinion, what are the key research directions to further improve the generalization and calibration capabilities of BDL in real-world distribution-shifted data scenarios?\ -How do frequentist methods like conformal prediction factor into this study?

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

See the weaknesses section above. -Figures 3 and 4 are a bit too small, consider using a horizontal legend for example.

Summary

The paper conducts a large-scale benchmark of Bayesian deep learning (BDL) methods for distribution-shifted data. The focus is on evaluating the quality of the posterior approximation, generalization ability, and calibration using signed versions of calibration metrics to distinguish between under- and overconfidence. The BDL algorithms are evaluated on convolutional and transformer-based architectures, mostly on regression and classification tasks from the WILDS collection. For most tasks, the experiments show improved generalization and calibration when extending single-mode approximations to multiple modes through ensembling. The authors identify a limitation of ensemble-based methods when finetuning transformer-based language models. In this task, accuracy and calibration are not significantly improved over single-mode approximations.

Strengths

Although there are similar experimental surveys of BDL algorithms, the paper benchmarks a wider range of state-of-the-art BDL algorithms applied to modern neural network architectures. Considering the success of ensembling in applications and recent competitions, the choice of focusing on the evaluation of ensembling single-mode BDL algorithms is relevant. The lack of improvement in the generalization and calibration of such methods in transformer-based finetuning tasks is insightful. The introduced signed calibration metrics are useful for making the distinction between over- and underconfident models. The authors have included code with their submission, which is useful for reviewing the details of the experiments and algorithmic implementations. The experiments are described with sufficient clarity and detail.

Weaknesses

1) The paper focuses largely on finetuning tasks, with the exceptions of corrupted CIFAR-10 and the large-scale regression task. The limitation should be described in the “Limitations” section or amended by including further experiments of models that are trained from scratch. Some of the algorithms have specific weaknesses or strengths that become only evident in such settings. For instance, ensemble-based methods may become computationally restrictive. Methods like iVON, SVGD, and BBB that modify the training objective or introduce noise to regularize the training may lead to base models with improved generalization or calibration. When focusing on finetuning of pre-trained models, these effects are neglected to some extent. 2) The experiment on the RXRX1- WILDS datasets in Fig. 3 lacks results for several algorithms. The figure caption and main text do not justify this choice. The exception is the Laplace approximation, where underperformance is noted as a reason.

Questions

1) In the conclusion, the authors state that “[...] we demonstrated that BDL is in many cases competitive with algorithms that are specifically designed for o.o.d. generalization.” To justify that conclusion, additional comparisons to such algorithms, either from existing works or through experimental evaluation, should be added. Except for a hint to the WILDS leaderboard for the large-scale regression experiments, comparisons to specialized methods that are designed for o.o.d. generalization are missing from the main text. 2) One main result of the paper is that ensembling-based BDL methods do not systematically improve upon the single-mode counterparts in finetuning of transformers-based language models. The authors hypothesize that this finding may be due to the nature of the finetuning task. There, the initialization from a pre-trained model may lead to a lack of diversity among the ensemble members. Considering the results of the “Finetuning for CNNs” section, where ensemble methods yield clear improvements, the hypothesis is not fully supported. To clarify the reason for the lack of performance of ensemble-based methods, results on a second architecture in addition to DistillBERT would be insightful. Experimental results to identify the mechanism among task, architecture (BERT-specific?), training procedure, or others, would further strengthen the submission.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

N/A

Summary

This paper compares a variety of Bayesian Deep Learning methods on the WILDS dataset collection comprised of regression and classification tasks with the aim of evaluation generalization performance under distribution shifts. In addition to previously proposed single-mode approximation methods, the authors extend the methodology to sampling ensembles of these kinds of methods and show an improvement on various tasks. Additionally, the authors adapt existing calibration metrics to a signed version that is able to reflect under- or over confidence more directly. General comment: - As you state, your paper is in line with the work of (Band et al. 2021) which was submitted to and accepted at the Neurips Benchmark track. The [Neurips website](https://nips.cc/Conferences/2023/CallForDatasetsBenchmarks) states that it is suited among other criteria for "Systematic analyses of existing systems on novel datasets yielding important new insight." which to me perfectly describes this work and in conclusion seems to be a better fit than the main track.

Strengths

- extensive experiments across image classification and regression tasks as well as text based classification - released code can be run without errors and provides adequate comments about running the code - the methodology of the paper builds on established datasets and methods and provides a useful benchmark of BDL on more realistic datasets

Weaknesses

- I believe the paper could benefit from a small discussion about why Bayesian Deep Learning in general should be well suited to OOD tasks since Bayesian methods through Bayes rule assume that the test data follows the same distribution as the training data. A relevant discussion seems to be this paper for example (https://arxiv.org/abs/2110.06020) - I think the training/evaluation setup over the different folds and how those folds are created should be explained more explicitly - The wide variety of methods you compare is a great strength of the paper, however, I was wondering why you did not include Deep Kernel Learning (DKL) ([Wilson et al. 2015](https://arxiv.org/abs/1511.02222)) or an "improvement" of it like DUE ([Amersfoort et al. 2021](https://arxiv.org/abs/2102.11409)) since their experiments show good generalization performance on a variety of tasks and are also theoretically grounded in Gaussian Processes which many regard as a "gold standard" in Bayesian Machine Learning and UQ - you state that you provide the first systematic evaluation of BDL for finetuning large pre-trained models, but I would argue that the common practice of using imagenet weights for example is also finetuning? Or are you saying your experiments are the first systematic evaluation with respect to binary text classification? - the performance of "generalizability" seems to focus on accuracy metrics such as the pearson coefficient or F1 score for classification, however, isn't a more suitable metric the NLL or other proper scoring rules [Gneiting and Raferty 2007](https://www.tandfonline.com/doi/abs/10.1198/016214506000001437) because accuracy metrics ignore the predictive uncertainty? I am aware that these are provided for the regression task in the appendix but should they not be part of the main analysis? Extra comments: - On some tables in the Appendix I am not clear about the use of bold face, for example in Table 8 the entire MSE column is bold, or I misread the table?

Questions

- I am slightly confused by how the methodology and code defines a predictive distribution or predictive uncertainty. It appears that you use a Gaussian Mixture Model approach for the Deep Ensembles, but in the appendix you state that "We optimize the log likelihood of the training data and use a fixed standard deviation of 0.1, as this is the value MAP converges to when jointly optimizing the standard deviation and the model's parameters". Is this supposed to be an estimate of the aleatoric uncertainty that is added to the epistemic uncertainty you obtain when computing a variance over the sampled point predictions? - Why can a pretrained model like resnet18 not be used for the povertyMAP task, but instead you train a model from scratch? At least I was not able to find loading an image-net pretrained weights in the code and there is lots of papers in the earth observation domain that show that image-net outperforms training from scratch even on satellite based optical imagery (as a latest example [Corley et al 2023](https://arxiv.org/pdf/2305.13456.pdf)) - from my understanding a strength of the Laplace approximation is that it yields an uncertainty estimate on top of the MAP estimate and it should theoretically yield the same mean prediction as the MAP model which should be reflected in the same accuracy metric, however, this is not the case in almost all experiments, like Figure 2 and 3. I understand that you are using a sampling approach but perhaps you are not using enough samples? A striking example seems Figure 3 where you state that Laplace underperforms on FMoW and RxRx1 while a Deep Ensemble that is also based on MAP estimates performs well. - As you state in your conclusion, BBB performs quiet well on a variety of tasks but it appears they are always beat by the much simpler MC-Dropout approach in terms of the TV metric when compared to the gold standard HMC uncertainty estimation (Figure 5). I was wondering whether you could comment on this discrepancy and whether you have a possible explanation for this?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The limitations are accurately addressed.

Response to Authors

I thank the authors for the response. I believe the authors addressed some of the concerns well (including mine) while less so to others, such as the need for a proper analysis as to why and when a certain method is preferred (which was also raised by Reviewer eT5S). Overall, I believe that this paper makes a nice contribution to the community and therefore I retain my original score.

Response to the Rebuttal

I thank the authors for the detailed response, I am happy with the answers and retain my original score. I would only like to re-emphasize the need to include a list of easily digestible takeaway findings in the discussion/conclusion section of the paper.