Schoenberg characterization of continuous non-stationary isotropic positive definite kernels

We characterize the continuous isotropic positive definite kernels on $\mathbb{R}^d$, where isotropy refers to invariance under the orthogonal group $O(d)$ but not necessarily stationarity. Furthermore, we characterize strict positive definiteness for such kernels. The class of isotropic kernels is fairly general as it unifies stationary isotropic and dot product kernels, and includes neural network kernels that arise from infinite-width limits of neural networks. As an application, we further characterize the continuous isotropic Gaussian random functions in terms of a series representation.