Approximations with deep neural networks in Sobolev time-space

Solutions of the evolution equation generally lie in certain Bochner–Sobolev spaces, in which the solutions may have regularity and integrability properties for the time variable that can be different for the space variables. Therefore, in this paper, we develop a framework that shows that deep neural networks can approximate Sobolev-regular functions with respect to Bochner–Sobolev spaces. In our work, we use the so-called Rectified Cubic Unit (ReCU) as an activation function in our networks. This activation function allows us to deduce approximation results of the neural networks while avoiding issues caused by the nonregularity of the most commonly used Rectified Linear Unit (ReLU) activation function.