Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networks

Most of existing statistical theories on deep neural networks have sample\ncomplexities cursed by the data dimension and therefore cannot well explain the\nempirical success of deep learning on high-dimensional data. To bridge this\ngap, we propose to exploit low-dimensional geometric structures of the real\nworld data sets. We establish theoretical guarantees of convolutional residual\nnetworks (ConvResNet) in terms of function approximation and statistical\nestimation for binary classification. Specifically, given the data lying on a\n$d$-dimensional manifold isometrically embedded in $\\mathbb{R}^D$, we prove\nthat if the network architecture is properly chosen, ConvResNets can (1)\napproximate Besov functions on manifolds with arbitrary accuracy, and (2) learn\na classifier by minimizing the empirical logistic risk, which gives an excess\nrisk in the order of $n^{-\\frac{s}{2s+2(s\\vee d)}}$, where $s$ is a smoothness\nparameter. This implies that the sample complexity depends on the intrinsic\ndimension $d$, instead of the data dimension $D$. Our results demonstrate that\nConvResNets are adaptive to low-dimensional structures of data sets.\n