UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning Leveraging Planning

Offline reinforcement learning (RL) provides a framework for learning\ndecision-making from offline data and therefore constitutes a promising\napproach for real-world applications as automated driving. Self-driving\nvehicles (SDV) learn a policy, which potentially even outperforms the behavior\nin the sub-optimal data set. Especially in safety-critical applications as\nautomated driving, explainability and transferability are key to success. This\nmotivates the use of model-based offline RL approaches, which leverage\nplanning. However, current state-of-the-art methods often neglect the influence\nof aleatoric uncertainty arising from the stochastic behavior of multi-agent\nsystems. This work proposes a novel approach for Uncertainty-aware Model-Based\nOffline REinforcement Learning Leveraging plAnning (UMBRELLA), which solves the\nprediction, planning, and control problem of the SDV jointly in an\ninterpretable learning-based fashion. A trained action-conditioned stochastic\ndynamics model captures distinctively different future evolutions of the\ntraffic scene. The analysis provides empirical evidence for the effectiveness\nof our approach in challenging automated driving simulations and based on a\nreal-world public dataset.\n