On the validity of kernel approximations for orthogonally-initialized neural networks

In this note we extend kernel function approximation results for neural networks with Gaussian-distributed weights to single-layer networks initialized using Haar-distributed random orthogonal matrices (with possible rescaling). This is accomplished using recent results from random matrix theory.

cs.LG cs.NE Stochastic Gradient Optimization Techniques Matrix Theory and Algorithms Neural Networks and Applications Computer Science Mathematics

View graph Source

Paper

Similar papers

Multi-layer random features and the approximation power of neural networksRustem Takhanov2024 · arXiv (Cornell University) · 4 citations In Library

Non-asymptotic approximations of neural networks by Gaussian processesRonen Eldan, Dan Mikulincer, T. Schramm2021 · Annual Conference Computational Learning Theory · 26 citations In Library

Approximation of RKHS Functionals by Neural NetworksTiancong Zhou, Namjoon Suh, Guang Cheng et al.2024 · arXiv.org · 6 citations In Library

Function Approximation with Randomly Initialized Neural Networks for Approximate Model Reference Adaptive ControlTyler Lekang, Andrew Lamperski2023 · arXiv (Cornell University) · 1 citation In Library

Universal approximation property of Banach space-valued random feature models including random neural networksAriel Neufeld, Philipp Schmocker2023 · Open MIND In Library

Universal approximation property of Banach space-valued random feature models including random neural networksAriel Neufeld, Philipp Schmocker2023 · arXiv (Cornell University) · 15 citations In Library

Approximation by non-symmetric networks for cross-domain learningH. Mhaskar2023 · Neural Networks · 2 citations In Library

Universal approximation results for neural networks with non-polynomial activation function over non-compact domainsAriel Neufeld, Philipp Schmocker2025 · Analysis and Applications In Library

Universal approximation results for neural networks with non-polynomial activation function over non-compact domainsAriel Neufeld, Philipp Schmocker2024 · arXiv (Cornell University) · 10 citations In Library

Training Neural Networks as Learning Data-adaptive Kernels: Provable Representation and Approximation BenefitsXialiang Dou, Tengyuan Liang2019 · Journal of the American Statistical Association · 42 citations In Library

Source of record

arXiv

44B

Welcome.

This panel sticks with you. Pick anything below and it opens right beside it, so you can dig through 60,000-plus records without ever losing your spot here.

Travel 44B

LibraryPapers, policy, standards, statute LabsEvery organization building AI ModelsIntelligence, price, and speed BenchmarksThe evaluation catalog IncidentsAI failures and harms SearchOne field across everything DashboardThe state of AI accountability in NY 44B RegistryThe Art. 44-B compliance portal