Asymptotic properties of one-layer artificial neural networks with sparse connectivity

A law of large numbers for the empirical distribution of parameters of a one-layer artificial neural networks with sparse connectivity is derived for a simultaneously increasing number of both, neurons and training iterations of the stochastic gradient descent.

cond-mat.dis-nn math.PR stat.ML Neural Networks and Applications Computer Science Mathematics Physics

View graph Source

Paper

References (19)

01Influence of random topology in artificial neural networks: A surveySara Kaviani, I. Sohn2020 · ICT express · 23 citations

02Fluctuation Theory in the Boltzmann–Grad LimitT. Bodineau, I. Gallagher, L. Saint-Raymond et al.2020 · Journal of statistical physics · 40 citations

03Pre-Defined Sparse Neural Networks With Hardware AccelerationSourya Dey, Kuan-Wen Huang, P. Beerel et al.2018 · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · 27 citations In Library

04Mean field analysis of neural networks: A central limit theoremJustin A. Sirignano, K. Spiliopoulos2018 · Stochastic Processes and their Applications · 219 citations In Library

05On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal TransportLénaïc Chizat, F. Bach2018 · Neural Information Processing Systems · 859 citations In Library

06Mean Field Analysis of Neural Networks: A Law of Large NumbersJustin A. Sirignano, K. Spiliopoulos2018 · SIAM Journal on Applied Mathematics · 245 citations In Library

07A mean field view of the landscape of two-layer neural networksSong Mei, A. Montanari, Phan-Minh Nguyen2018 · Proceedings of the National Academy of Sciences of the United States of America · 1.0k citations In Library

08Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network scienceD. Mocanu, Elena Mocanu, P. Stone et al.2017 · Nature Communications · 754 citations In Library

09Reaction–Diffusion models: From particle systems to SDE’sConrado da Costa, Bernardo Freitas Paulo da Costa, M. Jara2017 · Stochastic Processes and their Applications · 4 citations

10Understanding brain networks and brain organizationL. Pessoa2014 · Physics of Life Reviews · 433 citations

11Nonlinear Markov Processes and Kinetic EquationsV. Kolokoltsov2010 · 320 citations

12Principles and Theory for Data Mining and Machine LearningB. Clarke, Ernest Fokoue, H. Zhang2009 · 332 citations

Scroll for more · 7 remaining

Similar papers

Mean Field Analysis of Neural Networks: A Law of Large NumbersJustin A. Sirignano, K. Spiliopoulos2018 · SIAM Journal on Applied Mathematics · 241 citations In Library

Asymptotic description of stochastic neural networks. I. Existence of a large deviation principleO. Faugeras, J. MacLaurin2014 · Comptes Rendus Mathematique · 12 citations In Library

Asymptotic Description of Neural Networks with Correlated Synaptic WeightsO. Faugeras, J. MacLaurin2013 · Entropy · 21 citations In Library

Asymmetric Scaling Laws from Sparse FeaturesJ. Sous, Mike Winer2026 · 0 citations In Library

Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network ScienceD. Mocanu, Elena Mocanu, P. Stone et al.2017 · Nature Communications · 750 citations In Library

Random Sparse Lifts: Construction, Analysis and Convergence of finite sparse networksDavid A. R. Robin, Kevin Scaman, Marc Lelarge2025 · International Conference on Learning Representations · 0 citations In Library

Mean Field Analysis of Deep Neural NetworksJustin A. Sirignano, K. Spiliopoulos2019 · Mathematics of Operations Research · 94 citations In Library

Law of large numbers and central limit theorem for wide two-layer neural networks: the mini-batch and noisy caseArnaud Descours, Arnaud Guillin, Manon Michel et al.2022 · arXiv (Cornell University) · 6 citations In Library

Asymptotic Properties of Neural Network Sieve EstimatorsXiaoxi Shen, Chang Jiang, Lyudamila Sakhanenko et al.2019 · Journal of nonparametric statistics (Print) · 22 citations In Library

Generalization of Two-layer Neural Networks: An Asymptotic ViewpointJimmy Ba, Murat A. Erdogdu, Taiji Suzuki et al.2020 · International Conference on Learning Representations In Library

Source of record

arXiv

44B

Welcome.

This panel sticks with you. Pick anything below and it opens right beside it, so you can dig through 60,000-plus records without ever losing your spot here.

Travel 44B

LibraryPapers, policy, standards, statute LabsEvery organization building AI ModelsIntelligence, price, and speed BenchmarksThe evaluation catalog IncidentsAI failures and harms SearchOne field across everything DashboardThe state of AI accountability in NY 44B RegistryThe Art. 44-B compliance portal