A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks

We present a generalization bound for feedforward neural networks in terms of the product of the spectral norm of the layers and the Frobenius norm of the weights. The generalization bound is derived using a PAC-Bayes analysis.

cs.LG Computer Science Mathematics

Paper

Similar papers

Spectrally-normalized margin bounds for neural networksP. Bartlett, Dylan J. Foster, Matus Telgarsky2017 · Neural Information Processing Systems · 1.5k citations In Library

PAC-Bayesian Margin Bounds for Convolutional Neural NetworksK. Pitas, M. Davies, P. Vandergheynst2017 · arXiv.org · 13 citations In Library

PAC-Bayesian Spectrally-Normalized Bounds for Adversarially Robust GeneralizationJiancong Xiao, Ruoyu Sun, Zhi‐Quan Luo2023 · arXiv (Cornell University) · 12 citations In Library

On the Importance of Gradient Norm in PAC-Bayesian BoundsItai Gat, Yossi Adi, Alexander G. Schwing et al.2022 · arXiv (Cornell University) · 7 citations In Library

A PAC-Bayesian Approach to Generalization Bounds for Graph Neural NetworksRenjie Liao, Raquel Urtasun, Richard S. Zemel2020 · arXiv (Cornell University) · 113 citations In Library

Robust Large Margin Deep Neural NetworksJure Sokolić, R. Giryes, G. Sapiro et al.2016 · IEEE Transactions on Signal Processing · 349 citations In Library

Non-Vacuous Generalisation Bounds for Shallow Neural NetworksFelix Biggs, Benjamin Guedj2022 · arXiv (Cornell University) · 28 citations In Library

Improving Generalization of Deep Neural Networks by Leveraging Margin DistributionShen-Huan Lyu, Lu Wang, Zhi-Hua Zhou2018 · Neural Networks · 15 citations In Library

Deterministic PAC-Bayesian generalization bounds for deep networks via generalizing noise-resilienceVaishnavh Nagarajan, J. Kolter2019 · International Conference on Learning Representations · 105 citations In Library

Demystify Optimization and Generalization of Over-parameterized PAC-Bayesian LearningWei Huang, Chunrui Liu, Yilan Chen et al.2022 · arXiv (Cornell University) In Library

Source of record

arXiv

44B

Welcome.

This panel sticks with you. Pick anything below and it opens right beside it, so you can dig through 60,000-plus records without ever losing your spot here.

Travel 44B

LibraryPapers, policy, standards, statute LabsEvery organization building AI ModelsIntelligence, price, and speed BenchmarksThe evaluation catalog IncidentsAI failures and harms SearchOne field across everything DashboardThe state of AI accountability in NY 44B RegistryThe Art. 44-B compliance portal