Approximation Error of Sobolev Regular Functions with Tanh Neural Networks: Theoretical Impact on PINNs

Abstract

. Considering the key role played by derivatives in Partial Differential Equations (PDEs), using the tanh activation function in Physics-Informed Neural Networks (PINNs) yields useful smoothness properties to derive theoretical guarantees in Sobolev norm. In this paper, we conduct an extensive functional analysis, unveiling tighter approximation bounds compared to prior works, especially for higher order PDEs. These better guarantees translate into smaller PINN architectures and improved generalization error with arbitrarily small Sobolev norms of the PDE residuals.