Soft Actor-Critic with Backstepping-Pretrained DeepONet for control of PDEs

This paper develops a reinforcement learning-based controller for the stabilization of partial differential equation (PDE) systems. Within the Soft Actor-Critic (SAC) framework, we embed a DeepONet, a well-known neural operator (NO), which is pretrained using the backstepping controller. The pretrained DeepONet captures the essential features of the backstepping controller and is used to extract features, replacing the convolutional neural networks (CNNs) in the original actor-critic networks, and directly connects to the fully connected layers of the SAC architecture. We apply this novel backstepping and reinforcement learning integrated method to stabilize an unstable 1D hyperbolic PDE and an unstable reaction-diffusion PDE. The proposed method is shown to outperform standard SAC in simulations, SAC with an untrained DeepONet, and the backstepping controller on both systems.