Deep neural networks (DNNs) have achieved state-of-the-art performances in\nmany important domains, including medical diagnosis, security, and autonomous\ndriving. In these domains where safety is highly critical, an erroneous\ndecision can result in serious consequences. While a perfect prediction\naccuracy is not always achievable, recent work on Bayesian deep networks shows\nthat it is possible to know when DNNs are more likely to make mistakes. Knowing\nwhat DNNs do not know is desirable to increase the safety of deep learning\ntechnology in sensitive applications. Bayesian neural networks attempt to\naddress this challenge. However, traditional approaches are computationally\nintractable and do not scale well to large, complex neural network\narchitectures. In this paper, we develop a theoretical framework to approximate\nBayesian inference for DNNs by imposing a Bernoulli distribution on the model\nweights. This method, called MC-DropConnect, gives us a tool to represent the\nmodel uncertainty with little change in the overall model structure or\ncomputational cost. We extensively validate the proposed algorithm on multiple\nnetwork architectures and datasets for classification and semantic segmentation\ntasks. We also propose new metrics to quantify the uncertainty estimates. This\nenables an objective comparison between MC-DropConnect and prior approaches.\nOur empirical results demonstrate that the proposed framework yields\nsignificant improvement in both prediction accuracy and uncertainty estimation\nquality compared to the state of the art.\n