Reinforcement Learning with Uncertainty Estimation for Tactical Decision-Making in Intersections

This paper investigates how a Bayesian reinforcement learning method can be\nused to create a tactical decision-making agent for autonomous driving in an\nintersection scenario, where the agent can estimate the confidence of its\nrecommended actions. An ensemble of neural networks, with additional randomized\nprior functions (RPF), are trained by using a bootstrapped experience replay\nmemory. The coefficient of variation in the estimated $Q$-values of the\nensemble members is used to approximate the uncertainty, and a criterion that\ndetermines if the agent is sufficiently confident to make a particular decision\nis introduced. The performance of the ensemble RPF method is evaluated in an\nintersection scenario, and compared to a standard Deep Q-Network method. It is\nshown that the trained ensemble RPF agent can detect cases with high\nuncertainty, both in situations that are far from the training distribution,\nand in situations that seldom occur within the training distribution. In this\nstudy, the uncertainty information is used to choose safe actions in unknown\nsituations, which removes all collisions from within the training distribution,\nand most collisions outside of the distribution.\n