Implicit Weight Uncertainty in Neural Networks

We interpret HyperNetworks within the framework of variational inference within implicit distributions. Our method, Bayes by Hypernet, is able to model a richer variational distribution than previous methods. Experiments show that it achieves comparable predictive performance on the MNIST classification task while providing higher predictive uncertainties compared to MC-Dropout and regular maximum likelihood training.

ML methods Security & robustness Vision stat.ML cs.LG Adversarial Robustness in Machine Learning Anomaly Detection Techniques and Applications Generative Adversarial Networks and Image Synthesis Computer Science Mathematics

View graph Source

Paper

References (16)

01Bayesian HypernetworksDavid Krueger, Chin-Wei Huang, Riashat Islam et al.2017 · arXiv.org · 158 citations In Library

02SMASH: One-Shot Model Architecture Search through HyperNetworksAndrew Brock, Theodore Lim, J. Ritchie et al.2017 · International Conference on Learning Representations · 808 citations In Library

03On Calibration of Modern Neural NetworksChuan Guo, Geoff Pleiss, Yu Sun et al.2017 · International Conference on Machine Learning · 9.0k citations In Library

04Multiplicative Normalizing Flows for Variational Bayesian Neural NetworksChristos Louizos, M. Welling2017 · International Conference on Machine Learning · 493 citations In Library

05Deep and Hierarchical Implicit ModelsDustin Tran, R. Ranganath, D. Blei2017 · arXiv.org · 101 citations In Library

06Variational Inference using Implicit DistributionsFerenc Huszár2017 · arXiv.org · 145 citations In Library

07Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial NetworksL. Mescheder, Sebastian Nowozin, Andreas Geiger2017 · International Conference on Machine Learning · 573 citations In Library

08HyperNetworksDavid Ha, Andrew M. Dai, Quoc V. Le2016 · International Conference on Learning Representations · 2.0k citations In Library

09Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural NetworksTim Salimans, Diederik P. Kingma2016 · Neural Information Processing Systems · 2.1k citations In Library

10Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational InferenceY. Gal, Zoubin Ghahramani2015 · arXiv.org · 829 citations In Library

11Weight Uncertainty in Neural NetworksC. Blundell, Julien Cornebise, K. Kavukcuoglu et al.2015 · arXiv.org · 2.2k citations In Library

12Probabilistic Backpropagation for Scalable Learning of Bayesian Neural NetworksJosé Miguel Hernández-Lobato, Ryan P. Adams2015 · International Conference on Machine Learning · 1.0k citations In Library

Scroll for more · 4 remaining

Similar papers

Weight Uncertainty in Neural NetworksC. Blundell, Julien Cornebise, K. Kavukcuoglu et al.2015 · arXiv.org · 2.2k citations In Library

Measuring Uncertainty through Bayesian Learning of Deep Neural Network StructureZhijie Deng, Yucen Luo, Jun Zhu et al.2019 · arXiv.org · 2 citations In Library

Quantification of Uncertainties in Probabilistic Deep Neural Network by Implementing Boosting of Variational InferencePavia Bera, S. Bhanja2025 · arXiv.org · 2 citations In Library

Correlated Parameters to Accurately Measure Uncertainty in Deep Neural NetworksKonstantin Posch, J. Pilz2019 · IEEE Transactions on Neural Networks and Learning Systems · 32 citations In Library

Correlated Parameters to Accurately Measure Uncertainty in Deep Neural NetworksKonstantin Posch, Jürgen Pilz2020 · IEEE Transactions on Neural Networks and Learning Systems In Library

Measuring the Uncertainty of Predictions in Deep Neural Networks with Variational InferenceJan Steinbrener, Konstantin Posch, Jürgen Pilz2020 · Sensors · 18 citations In Library

Minimizing the weight uncertainty in Neural Networks Using Bayesian OptimizationPriyangshu Sarkar, U. Hariharan2023 In Library

Projected BNNs: Avoiding weight-space pathologies by learning latent representations of neural network weightsM. Pradier, Weiwei Pan, Jiayu Yao et al.2018 · 10 citations In Library

Variational Neural NetworksIllia Oleksiienko, Dat Tran, Alexandros Iosifidis2023 · Procedia Computer Science In Library

Bayesian Uncertainty Estimation for Batch Normalized Deep NetworksMattias Teye, Hossein Azizpour, Kevin Smith2018 · International Conference on Machine Learning · 267 citations In Library

Source of record

arXiv

44B

Welcome.

This panel sticks with you. Pick anything below and it opens right beside it, so you can dig through 60,000-plus records without ever losing your spot here.

Travel 44B

LibraryPapers, policy, standards, statute LabsEvery organization building AI ModelsIntelligence, price, and speed BenchmarksThe evaluation catalog IncidentsAI failures and harms SearchOne field across everything DashboardThe state of AI accountability in NY 44B RegistryThe Art. 44-B compliance portal