Application of deep neural networks to medical imaging tasks has in some\nsense become commonplace. Still, a "thorn in the side" of the deep learning\nmovement is the argument that deep networks are prone to overfitting and are\nthus unable to generalize well when datasets are small (as is common in medical\nimaging tasks). One way to bolster confidence is to provide mathematical\nguarantees, or bounds, on network performance after training which explicitly\nquantify the possibility of overfitting. In this work, we explore recent\nadvances using the PAC-Bayesian framework to provide bounds on generalization\nerror for large (stochastic) networks. While previous efforts focus on\nclassification in larger natural image datasets (e.g., MNIST and CIFAR-10), we\napply these techniques to both classification and segmentation in a smaller\nmedical imagining dataset: the ISIC 2018 challenge set. We observe the\nresultant bounds are competitive compared to a simpler baseline, while also\nbeing more explainable and alleviating the need for holdout sets.\n