When to Localize? A Risk-Constrained Reinforcement Learning Approach

In robotic navigation, continuous localization can be inefficient in certain scenarios, such as underwater searches. For example, an underwater agent surfacing to localize too often hinders it from searching for critical items underwater, such as black boxes from crashed aircraft. On the other hand, if the agent never localizes, it may fail to find the items due to inadvertently leaving the search area or entering hazardous, restricted areas. Motivated by such scenarios, we explore approaches to help an agent determine "when to localize?". We propose a bi-criteria optimization approach to determine optimal localization timing, which balances localization frequency with failure probability. Our method, RiskRL, employs a Particle Filter and a constrained Reinforcement Learning (RL) framework, which uses a recurrent Soft Actor-Critic (SAC) network. This approach improves our previous POMDP-based solution by reducing computational complexity and eliminating the need for complete transition and observation models. Experimental results demonstrate that RiskRL showed a 26% increase in success rates in unseen environments over our baselines. The implementation is available on GitHub: https://github.com/raaslab/when-to-localize-riskrl