Reachability-based Trajectory Safeguard (RTS): A Safe and Fast Reinforcement Learning Safety Layer for Continuous Control

Reinforcement Learning (RL) algorithms have achieved remarkable performance\nin decision making and control tasks due to their ability to reason about\nlong-term, cumulative reward using trial and error. However, during RL\ntraining, applying this trial-and-error approach to real-world robots operating\nin safety critical environment may lead to collisions. To address this\nchallenge, this paper proposes a Reachability-based Trajectory Safeguard (RTS),\nwhich leverages reachability analysis to ensure safety during training and\noperation. Given a known (but uncertain) model of a robot, RTS precomputes a\nForward Reachable Set of the robot tracking a continuum of parameterized\ntrajectories. At runtime, the RL agent selects from this continuum in a\nreceding-horizon way to control the robot; the FRS is used to identify if the\nagent's choice is safe or not, and to adjust unsafe choices. The efficacy of\nthis method is illustrated on three nonlinear robot models, including a 12-D\nquadrotor drone, in simulation and in comparison with state-of-the-art safe\nmotion planning methods.\n