Tactical Decision-Making in Autonomous Driving by Reinforcement Learning with Uncertainty Estimation

Reinforcement learning (RL) can be used to create a tactical decision-making\nagent for autonomous driving. However, previous approaches only output\ndecisions and do not provide information about the agent's confidence in the\nrecommended actions. This paper investigates how a Bayesian RL technique, based\non an ensemble of neural networks with additional randomized prior functions\n(RPF), can be used to estimate the uncertainty of decisions in autonomous\ndriving. A method for classifying whether or not an action should be considered\nsafe is also introduced. The performance of the ensemble RPF method is\nevaluated by training an agent on a highway driving scenario. It is shown that\nthe trained agent can estimate the uncertainty of its decisions and indicate an\nunacceptable level when the agent faces a situation that is far from the\ntraining distribution. Furthermore, within the training distribution, the\nensemble RPF agent outperforms a standard Deep Q-Network agent. In this study,\nthe estimated uncertainty is used to choose safe actions in unknown situations.\nHowever, the uncertainty information could also be used to identify situations\nthat should be added to the training process.\n