Deep learning approaches to surrogates for solving the diffusion equation for mechanistic real-world simulations

In many mechanistic medical, biological, physical and engineered\nspatiotemporal dynamic models the numerical solution of partial differential\nequations (PDEs) can make simulations impractically slow. Biological models\nrequire the simultaneous calculation of the spatial variation of concentration\nof dozens of diffusing chemical species. Machine learning surrogates, neural\nnetworks trained to provide approximate solutions to such complicated numerical\nproblems, can often provide speed-ups of several orders of magnitude compared\nto direct calculation. PDE surrogates enable use of larger models than are\npossible with direct calculation and can make including such simulations in\nreal-time or near-real time workflows practical. Creating a surrogate requires\nrunning the direct calculation tens of thousands of times to generate training\ndata and then training the neural network, both of which are computationally\nexpensive. We use a Convolutional Neural Network to approximate the stationary\nsolution to the diffusion equation in the case of two equal-diameter, circular,\nconstant-value sources located at random positions in a two-dimensional square\ndomain with absorbing boundary conditions. To improve convergence during\ntraining, we apply a training approach that uses roll-back to reject stochastic\nchanges to the network that increase the loss function. The trained neural\nnetwork approximation is about 1e3 times faster than the direct calculation for\nindividual replicas. Because different applications will have different\ncriteria for acceptable approximation accuracy, we discuss a variety of loss\nfunctions and accuracy estimators that can help select the best network for a\nparticular application.\n