Statistical learning by sparse deep neural networks

We consider a deep neural network estimator based on empirical risk minimization with l_1-regularization. We derive a general bound for its excess risk in regression and classification (including multiclass), and prove that it is adaptively nearly-minimax (up to log-factors) simultaneously across the entire range of various function classes.

math.ST cs.LG stat.ME stat.ML Computer Science Mathematics

Paper

References (33)

01Statistical guarantees for sparse deep learningJohannes Lederer2022 · AStA Advances in Statistical Analysis · 16 citations In Library

02Upper bounds on the Natarajan dimensions of some function classesYingji Jin2022 · International Symposium on Information Theory · 13 citations In Library

03On the rate of convergence of fully connected deep neural network regression estimatesMichael Kohler, S. Langer2021 · Annals of Statistics · 182 citations

04Exponential ReLU DNN Expression of Holomorphic Maps in High DimensionJ. Opschoor, C. Schwab, J. Zech2021 · Constructive approximation · 132 citations

05Deep learning: a statistical viewpointP. Bartlett, A. Montanari, A. Rakhlin2021 · Acta Numerica · 342 citations In Library

06Advantage of Deep Neural Networks for Estimating Functions with Singularity on HypersurfacesM. Imaizumi, K. Fukumizu2020 · Journal of machine learning research · 24 citations In Library

07Expressivity of Deep Neural NetworksIngo Gühring, Mones Raslan, Gitta Kutyniok2020 · arXiv.org · 65 citations In Library

08Statistical Guarantees for Regularized Neural NetworksMahsa Taheri, Fang Xie, Johannes Lederer2020 · Neural Networks · 43 citations In Library

09Nonconvex Sparse Regularization for Deep Neural Networks and Its OptimalityIlsang Ohn, Yongdai Kim2020 · Neural Computation · 29 citations In Library

10Fast convergence rates of deep neural networks for classificationYongdai Kim, Ilsang Ohn, Dongha Kim2018 · Neural Networks · 103 citations In Library

11Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionalityTaiji Suzuki2018 · International Conference on Learning Representations · 302 citations In Library

12Optimal approximation of continuous functions by very deep ReLU networksD. Yarotsky2018 · Annual Conference Computational Learning Theory · 340 citations In Library

Scroll for more · 21 remaining

Similar papers

Nearly Optimal Learning using Sparse Deep ReLU Networks in Regularized Empirical Risk Minimization with Lipschitz LossHuang, Ke, Mingming Liu, Ma, Shujie2021 · arXiv (Cornell University) · 1 citation In Library

Nearly Optimal Learning Using Sparse Deep ReLU Networks in Regularized Empirical Risk Minimization With Lipschitz LossKe Huang, Mingming Liu, Shujie Ma2025 · Neural Computation In Library

Nonconvex sparse regularization for deep neural networks and its optimalityIlsang Ohn, Yongdai Kim2020 · Neural Computation · 28 citations In Library

Deep learning from strongly mixing observations: Sparse-penalized regularization and minimax optimalityW. Kengne, M. Wade2024 · Journal of Complexity · 4 citations In Library

Sparse-penalized deep neural networks estimator under weak dependenceWilliam Kengne, M. Wade2023 · Metrika (Heidelberg) · 7 citations In Library

Error bounds for sparse classifiers in high-dimensionsA. Dedieu2018 · International Conference on Artificial Intelligence and Statistics · 8 citations In Library

Optimal Nonparametric Inference via Deep Neural NetworkRuiqi Liu, B. Boukai, Zuofeng Shang2019 · arXiv.org · 21 citations In Library

Approximation and Estimation for High-Dimensional Deep Learning NetworksA. Barron, Jason M. Klusowski2018 · arXiv.org · 63 citations In Library

Minimax optimality of deep neural networks on dependent data via PAC-Bayes boundsPierre Alquier, William Kengne2025 · Electronic Journal of Statistics In Library

On the minimax optimality and superiority of deep neural network learning over sparse parameter spacesSatoshi Hayakawa, Taiji Suzuki2019 · Neural Networks · 57 citations In Library

Source of record

arXiv

44B

Welcome.

This panel sticks with you. Pick anything below and it opens right beside it, so you can dig through 60,000-plus records without ever losing your spot here.

Travel 44B

LibraryPapers, policy, standards, statute LabsEvery organization building AI ModelsIntelligence, price, and speed BenchmarksThe evaluation catalog IncidentsAI failures and harms SearchOne field across everything DashboardThe state of AI accountability in NY 44B RegistryThe Art. 44-B compliance portal