The Representation Power of Neural Networks: Breaking the Curse of Dimensionality

In this paper, we analyze the number of neurons and training parameters that\na neural networks needs to approximate multivariate functions of bounded second\nmixed derivatives -- Korobov functions. We prove upper bounds on these\nquantities for shallow and deep neural networks, breaking the curse of\ndimensionality. Our bounds hold for general activation functions, including\nReLU. We further prove that these bounds nearly match the minimal number of\nparameters any continuous function approximator needs to approximate Korobov\nfunctions, showing that neural networks are near-optimal function\napproximators.\n