Analysis of the rate of convergence of fully connected deep neural\n network regression estimates with smooth activation function

This article contributes to the current statistical theory of deep neural\nnetworks (DNNs). It was shown that DNNs are able to circumvent the so--called\ncurse of dimensionality in case that suitable restrictions on the structure of\nthe regression function hold. In most of those results the tuning parameter is\nthe sparsity of the network, which describes the number of non-zero weights in\nthe network. This constraint seemed to be the key factor for the good rate of\nconvergence results. Recently, the assumption was disproved. In particular, it\nwas shown that simple fully connected DNNs can achieve the same rate of\nconvergence. Those fully connected DNNs are based on the unbounded ReLU\nactivation function. In this article we extend the results to smooth activation\nfunctions, i.e., to the sigmoid activation function. It is shown that\nestimators based on fully connected DNNs with sigmoid activation function also\nachieve the minimax rates of convergence (up to $\\ln n$-factors). In our result\nthe number of hidden layers is fixed, the number of neurons per layer tends to\ninfinity for sample size tending to infinity and a bound for the weights in the\nnetwork is given.\n